Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Agon, a novel reinforcement learning framework where two competing models serve as implicit graders for each other's reasoning processes, enhancing their ability to solve complex problems. By alternating roles in drafting and evaluating solutions, Agon fosters a competitive environment that encourages deeper reasoning rather than merely producing more content. The results demonstrate a significant performance improvement on challenging benchmarks, achieving double the pass rate of traditional methods like GRPO on the DeepMath dataset, highlighting the effectiveness of this cross-model approach.
Competing models can implicitly grade each other's reasoning, leading to a dramatic performance boost in complex problem-solving tasks.
Reinforcement learning from verifiable rewards (e.g. GRPO) is the engine behind today's reasoning models, yet it grades only the final answer. On hard problems this trains models to write more rather than to think better, since the trace itself is never graded and no label for good thinking exists. We introduce Agon, which makes two competing models each other's graders. Both attempt the same problem; in alternating roles, one drafts a solution and the other reads it while solving, and each is rewarded for out-solving the other. To win, a model must out-reason a rival that has seen its work, so reasoning is judged implicitly during training, with no process labels and no reward model. Because both models are optimized, each faces a progressively stronger rival, which single-model RL cannot provide. The two need only be comparably strong and behaviorally different. At inference the pair deploys as it trains, a two-stage cascade in which one model drafts and the other answers after reading the draft. On the hard split of DeepMath with Qwen3, this doubles GRPO's pass@1, roughly eight times the gain of an untrained Mixture-of-Agents pass over the same base. The ordering replicates on competitive-programming code and across model families (Qwen3.5, Gemma 4). For now the models talk in text; the next step is to let them reason together in latent space.