Search papers, labs, and topics across Lattice.
This paper introduces EvoTrainer, a novel framework that co-evolves LLM policies and their training harnesses using empirical feedback to enhance autonomous reinforcement learning. By addressing the limitations of static training setups, EvoTrainer effectively diagnoses and revises interventions, leading to superior performance in mathematical reasoning, competitive programming, and software engineering tasks. The results demonstrate that EvoTrainer not only matches but often exceeds traditional human-engineered RL benchmarks, particularly excelling in long-horizon agentic software engineering tasks.
EvoTrainer reveals that co-evolving LLM policies with their training harnesses can significantly outperform static approaches, especially in complex tasks like software engineering.
Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static. This limitation sharpens in agentic RL, where shifting bottlenecks and scalar rewards mask diverse failure modes. We introduce EvoTrainer, an autonomous training framework that co-evolves LLM policies and training-side harnesses through empirical feedback: it diagnoses rollout-level evidence, revises diagnostics, backtests interventions, and accumulates reusable skills. Evaluated on mathematical reasoning, competitive-programming code generation, and repository-level software engineering, EvoTrainer matches or exceeds the human-engineered RL references under the same data, codebase, and evaluation protocol, with the largest gain on long-horizon agentic SWE. Trajectory analyses show that retained strategies diverge across domains, evolving diagnostics prevent invalid high-scoring branches from being promoted, and reusable skills shape later search. Autonomous LLM RL should move beyond recipe search toward joint evolution of policies and the training harnesses that interpret them.
DAMO