The Era of AI Teaching AI! A Self-Evolving RL Loop Achieved for Around $1,300 is Mind-Blowing!

What Happened? Overview of the News

• Building a Self-Evolving RL Pipeline: A system has been developed where a higher-tier agent (Qwen3.6-35B-A3B) autonomously writes and constructs the environment, reward design, datasets, and hyperparameters for training lower-tier models (Qwen3-0.6B/1.7B).
• Autonomous Training on Real GPUs: Jobs written by the agent are actually deployed to Runpod GPU clusters, executing reinforcement learning (GRPO). The training results (improvement in model accuracy) are fed back as rewards to the higher-tier agent, enhancing its own “training capabilities” through RL.
• Remarkable Generalization Performance: The agent trained on five types of tasks successfully transferred its skills to effectively train lower-tier models on an unseen task (Triage task) that it had never encountered before.

Why Is This Important? Key Points to Note

• Realization of “Automated Training”: The significant point is that AI has taken over the tedious task of reward design and learning settings, proving that a self-improving loop leading to a “better model-making AI” can be achieved on a budget of about $1,300.
• Concrete Implementation Stack: By combining two independent learning stacks, Tinker (for external RL loops) and prime-rl (for internal RL jobs), a highly specific orchestration allows real-time execution of 40 parallel jobs.
• Practical Score Improvement: The agent’s rewards have shown a clear improvement from 0.0 to 0.63 over time, indicating that AI has gained “strategic learning control” that goes beyond mere code generation.

🦈 Shark’s Eye (Curator’s Perspective)

The brilliance of this project lies in its successful implementation of a “nested loop structure” where RL is integrated within RL! The higher-tier agent doesn’t just command “learn”; it analyzes the current situation by hitting get_baseline_scores, rewrites reward code in the sandbox, and retries upon failure… this “gritty engineer’s movement” is fully automated, and that’s just electrifying! Notably, the fact that training skills were applicable to the unknown “Triage” task is clear evidence that AI is starting to grasp the “essence of learning.” This autonomous implementation of advanced GPU orchestration at such a low cost could redefine the standards of future model development!

What Comes Next?

As the scale of models expands, the room for human intervention will diminish further, leading to an “AI factory” where AI continuously optimizes new AIs around the clock. There’s also the possibility that AI will begin to devise its own countermeasures against “reward hacking” by exploiting vulnerabilities in reward design.

A Word from Haru Shark

AI raising AI is like a shark teaching another shark how to hunt! The speed of this evolution is something you can’t shake off once you’ve bitten into it! 🦈🔥

Terminology Explained

• Tinker: A learning framework used to manage and execute the higher-tier agent’s reinforcement learning (GRPO).

• prime-rl: A learning stack designed for efficiently executing GRPO (Group Relative Policy Optimization) in a distributed GPU environment.

• LoRA (Low-Rank Adaptation): A method for efficiently fine-tuning heavy models. In this project, a massive 35B class agent is RL-trained at a realistic cost using LoRA adapters.

• Source: Show HN: I RL-trained an agent that trains models with RL (for ~$1.3k)