Search papers, labs, and topics across Lattice.
To prevent synthetic training environments from saturating as frontier agents improve, the authors introduce an off-policy environment evolution framework that continually scales task difficulty generation-by-generation. By deriving three core difficulty axes directly from the multi-turn learning objective, the system employs a loop-engineered multi-agent harness to systematically generate increasingly complex terminal environments independently of on-policy rollouts. Applying these evolved curricula to long-horizon RL yields substantial downstream gains, improving Qwen3.6-27B and Qwen3.6-35B-A3B by 14.4 and 18.0 percentage points on Terminal-Bench 2.1, respectively.
Decoupling task synthesis from on-policy rollouts solves the learning-signal saturation bottleneck in terminal agents, boosting long-horizon RL performance by up to 18 percentage points on Terminal-Bench 2.1.
Scaling interactive and verifiable environments is critical for training terminal agents. As frontier models become more capable, environments synthesized from scratch become less challenging and thus provide limited learning signals. Recent co-evolution methods iteratively synthesize environments near the model's learnable frontier based on weaknesses exposed during rollouts. However, their dependence on on-policy rollouts limits generalization and the continuous provision of learning signals as the model becomes stronger. In this paper, we propose environment evolution, which incrementally increases environment difficulty off-policy and schedules the evolved environments generation by generation during training to provide continuous learning signals. We derive three evolution directions that influence environment difficulty from the multi-turn learning objective and then implement evolution along these directions through a loop-engineered multi-agent harness. Quantitative rollout experiments with Hy4 preview, Claude Opus 5, and GPT-5.6 Sol show that environment evolution consistently produces more difficult environments. We validate its effectiveness on Qwen3.6-27B and Qwen3.6-35B-A3B through simple long-horizon RL training, improving their performance by 14.4 and 18.0 percentage points on Terminal-Bench 2.1, respectively.
