Search papers, labs, and topics across Lattice.
This paper introduces the task of Paper to Simulation Recoverability, which aims to automatically reconstruct control systems simulations from research papers. They curate a benchmark of 500 IEEE CDC papers and propose RESCORE, a three-component LLM agent (Analyzer, Coder, Verifier) that iteratively refines code using execution feedback and visual comparison. RESCORE achieves a 40.7% success rate in recovering task-coherent simulations, demonstrating a 10x speedup over manual replication.
LLM agents can now automatically reconstruct and verify control systems simulations from research papers, achieving a 10x speedup over manual replication.
Reconstructing numerical simulations from control systems research papers is often hindered by underspecified parameters and ambiguous implementation details. We define the task of Paper to Simulation Recoverability, the ability of an automated system to generate executable code that faithfully reproduces a paper's results. We curate a benchmark of 500 papers from the IEEE Conference on Decision and Control (CDC) and propose RESCORE, a three component LLM agentic framework, Analyzer, Coder, and Verifier. RESCORE uses iterative execution feedback and visual comparison to improve reconstruction fidelity. Our method successfully recovers task coherent simulations for 40.7% of benchmark instances, outperforming single pass generation. Notably, the RESCORE automated pipeline achieves an estimated 10X speedup over manual human replication, drastically cutting the time and effort required to verify published control methodologies. We will release our benchmark and agents to foster community progress in automated research replication.
