Search papers, labs, and topics across Lattice.
This paper introduces Critic Rubrics, a framework for learning a critic model from sparse and noisy human-agent interaction data by predicting behavioral features derived from interaction traces. A semi-supervised objective is used to jointly predict these rubrics and sparse human feedback. The resulting critic improves re-ranking on SWE-bench, enables early stopping, and supports training-time data curation.
Learn a critic for coding agents from human-in-the-loop interaction traces alone, sidestepping the need for dense, verifiable rewards.
Academic benchmarks for coding agents tend to reward autonomous task completion, measured by verifiable rewards such as unit-test success. In contrast, real-world coding agents operate with humans in the loop, where success signals are typically noisy, delayed, and sparse. How can we bridge this gap? In this paper, we propose a process to learn a "critic" model from sparse and noisy interaction data, which can then be used both as a reward model for either RL-based training or inference-time scaling. Specifically, we introduce Critic Rubrics, a rubric-based supervision framework with 24 behavioral features that can be derived from human-agent interaction traces alone. Using a semi-supervised objective, we can then jointly predict these rubrics and sparse human feedback (when present). In experiments, we demonstrate that, despite being trained primarily from trace-observable rubrics and sparse real-world outcome proxies, these critics improve best-of-N reranking on SWE-bench (Best@8 +15.9 over Random@8 over the rerankable subset of trajectories), enable early stopping (+17.7 with 83% fewer attempts), and support training-time data curation via critic-selected trajectories.
CMU ML