Search papers, labs, and topics across Lattice.
The paper introduces Hybrid-Gym, a synthetic training environment designed to improve the generalization capabilities of coding agents by focusing on transferable skills common across diverse software engineering tasks. They decompose successful coding trajectories into fine-grained components and use these to design auxiliary training tasks like function localization and dependency search. Training on Hybrid-Gym leads to significant performance gains on real-world benchmarks such as SWE-Bench Verified (25.4%), SWT-Bench Verified (7.9%), and Commit-0 Lite (5.1%), demonstrating effective generalization.
Forget training on narrow GitHub issues – Hybrid-Gym unlocks surprisingly broad coding skills by teaching agents to explore codebases and design architectures in synthetic environments.
When assessing the quality of coding agents, predominant benchmarks focus on solving single issues on GitHub, such as SWE-Bench. In contrast, in real use, these agents solve more various and complex tasks that involve other skills such as exploring codebases, testing software, and designing architecture. In this paper, we first characterize some transferable skills that are shared across diverse tasks by decomposing trajectories into fine-grained components, and derive a set of principles for designing auxiliary training tasks to teach language models these skills. Guided by these principles, we propose a training environment, Hybrid-Gym, consisting of a set of scalable synthetic tasks, such as function localization and dependency search. Experiments show that agents trained on our synthetic tasks effectively generalize to diverse real-world tasks that are not present in training, improving a base model by 25.4% absolute gain on SWE-Bench Verified, 7.9% on SWT-Bench Verified, and 5.1% on Commit-0 Lite. Hybrid-Gym also complements datasets built for the downstream tasks (e.g., improving SWE-Play by 4.9% on SWT-Bench Verified). Code available at: https://github.com/yiqingxyq/Hybrid-Gym.
CMU ML