Search papers, labs, and topics across Lattice.
The paper introduces Agon, a novel reinforcement learning framework where two competing models serve as each other's graders, fostering implicit evaluation of reasoning processes rather than solely focusing on final answers. This approach addresses the limitations of traditional reward systems that do not account for the reasoning trace, thereby encouraging models to enhance their cognitive capabilities in challenging problem-solving scenarios. Experimental results demonstrate that Agon significantly improves performance on the DeepMath benchmark, achieving double the pass rate of GRPO and outperforming untrained Mixture-of-Agents by a factor of eight.
Competing models in Agon implicitly grade each other's reasoning, leading to a dramatic boost in problem-solving performance without explicit process labels.
Reinforcement learning from verifiable rewards (e.g. GRPO) is the engine behind today's reasoning models, yet it grades only the final answer. On hard problems this trains models to write more rather than to think better, since the trace itself is never graded and no label for good thinking exists. We introduce Agon, which makes two competing models each other's graders. Both attempt the same problem; in alternating roles, one drafts a solution and the other reads it while solving, and each is rewarded for out-solving the other. To win, a model must out-reason a rival that has seen its work, so reasoning is judged implicitly during training, with no process labels and no reward model. Because both models are optimized, each faces a progressively stronger rival, which single-model RL cannot provide. The two need only be comparably strong and behaviorally different. At inference the pair deploys as it trains, a two-stage cascade in which one model drafts and the other answers after reading the draft. On the hard split of DeepMath with Qwen3, this doubles GRPO's pass@1, roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base. The ordering replicates on competitive-programming code and across model families (Qwen3.5, Gemma 4). For now the models talk in text; the next step is to let them reason together in latent space.
