Search papers, labs, and topics across Lattice.
This paper introduces a two-stage reinforcement learning framework, Test Cases Scaling (TCS), designed for the auto-generation of sound and adversarial test cases in code LLMs. By leveraging a rolling policy-aligned buffer, the first stage generates tests aligned with reference solutions, while the second stage focuses on generating counterexamples based on the model's current failure modes. The results show that TCS significantly enhances both pass rates and inference-time answer selection across benchmark datasets TACO and LiveCodeBench, demonstrating its effectiveness in improving code generation quality.
TCS not only generates sound test cases but also adapts to the model's weaknesses, leading to a marked improvement in code generation performance.
Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model. We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator also enables effective selection among other LLM outputs.
Tsinghua AI