Search papers, labs, and topics across Lattice.
To extend test-time reinforcement learning (TTRL) to code generation鈥攚here surface-form majority voting fails due to semantic equivalence across divergent syntaxes鈥攖he authors develop probe-driven behavioral consensus rewards paired with Entropy-Regularized Rank-Masked Policy Optimization (ERPO). ERPO safeguards against spurious consensus and reward hacking by converting low behavioral agreement into conservative negative gradient updates via rank masking while bounding policy drift with an entropy ceiling. Across competitive coding benchmarks, this framework delivers substantial pass@1 and pass@k improvements in both in-domain adaptation and zero-shot transfer settings without access to ground-truth verifiers.
Code generation models can now self-improve at test time without ground-truth unit tests by turning behavioral execution agreement on generated input probes into a stable, hack-resistant policy gradient signal.
Existing methods for test-time reinforcement learning (TTRL) derive rewards from answer-level self-voting on unlabeled test-time tasks with canonical answers, but this breaks down for code generation because programs cannot be compared by surface form and therefore do not directly provide a usable training signal. To make TTRL applicable to code generation, we propose probe-driven TTRL, which constructs output-free probe inputs from the problem statement, executes candidate programs on these probes, and defines a Probe Consensus Reward (PCR) from the resulting behavioral agreement. PCR provides a behavioral training signal for open-vocabulary programs, but it is not a fully reliable verifier and remains susceptible to reward hacking through spurious consensus. We therefore introduce Entropy-Regularized Rank-Masked Policy Optimization (ERPO), which converts low PCR into conservative negative updates through rank masking and controls policy drift with an entropy ceiling. On coding benchmarks, ERPO substantially improves pass@1 and pass@k in both in-domain adaptation and zero-shot transfer.
