Search papers, labs, and topics across Lattice.
This paper introduces VeriFine, an agent harness framework designed to overcome the limitations of fixed judges in self-improving embodied reasoning agents by enabling the co-evolution of the policy, training curriculum, and judge. Through a Policy Improvement Loop and a Judge Improvement Loop that incorporates human guidance and coactive calibration, VeriFine continuously refines both the agent's capabilities and its evaluation rubric. Experiments on driving and robot navigation demonstrate sustained self-improvement in both policy and judge performance, highlighting the efficacy of scaling verification for evolving failure patterns.
Embodied AI agents can now continuously self-improve by co-evolving their policies, training curricula, and judges, overcoming the bottleneck of static evaluation.
Self-improving policies continually expose new failure patterns, changing what their judges must be able to verify. However, current fixed judges constrain both optimization feedback and the discovery of useful training examples, limiting further self-improvement. This challenge is even more acute in embodied reasoning, where reliable evaluation must account for spatial grounding, causal reasoning, and safety-aware decision-making. We introduce VeriFine, an agent harness framework that scales verification through the co-evolution of the policy, training curriculum, and judge. The Policy Improvement Loop uses a rubric judge to diagnose recurring failures, construct an adaptive curriculum, and optimize the policy. When progress plateaus and verification becomes a bottleneck, the Judge Improvement Loop selectively queries human guidance on informative failure cases and refines the judge through coactive calibration, in which humans and agents resolve disagreements and converge toward the objective rubric of physical reasoning. The revised judge then guides the next stage of data selection and policy optimization. Experiments on driving and robot navigation tasks demonstrate continuous self-improvement in both policy and judge capability across reinforcement and supervised fine-tuning. These results show how scaling verification supports continuous self-improvement as policy failure patterns evolve.
BAIR