Search papers, labs, and topics across Lattice.
Change2Task is a novel system that transforms merged pull requests from software repositories into verified coding tasks, enhancing the training and evaluation of coding agents. By leveraging historical repository data and employing techniques like Patch Reversal and Code Mapping, it effectively reconstructs task states and validates the transition from a healthy codebase to executable tasks. The system demonstrates a 79.6% success rate in verified task construction and significantly outperforms existing baselines by recovering 29.2% more verified tasks, while also reducing pipeline costs by 10.8%.
Change2Task recovers 29.2% more verified coding tasks than traditional methods, streamlining the training of coding agents.
Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification. To expand this supply, we present Change2Task, a system grounded in repository history that converts merged pull requests into verified tasks on healthy modern revisions of the same repository. It aligns historical evidence with evolved code, reconstructs task states through Patch Reversal, Code Mapping, or Agent Reconstruction, and validates the lifecycle from a healthy base to a task state and a restored state. By deriving multiple tasks grounded in developer evidence from maintained environments, Change2Task provides executable data for coding agent training and evaluation while reducing repeated environment setup, storage, and task construction effort. We evaluate the system through five common and widely adopted coding agent task families: Bug Fix, Feature Addition, Test Generation, Application Programming Interface Migration, and Security Repair. Starting from 1,130 source changes eligible for construction, Change2Task achieves 79.6% verified task construction success across these task families. On a matched candidate set, it recovers 29.2% more verified tasks than a construction baseline based on pull requests. Historical and reconstructed cases achieve up to 98.0% matched outcome agreement under agent evaluation, while reuse of modern bases reduces measured expenditure across the complete pipeline by 10.8%.
Microsoft Research