Search papers, labs, and topics across Lattice.
This paper introduces a novel solver for the ARC-AGI-2 benchmark that leverages modality-driven search and holistic trace judging to enhance reasoning in large language models. By treating different reasoning modalities as independent search operators and employing a judge model to evaluate candidate traces collectively, the approach outperforms existing models, achieving a remarkable 72.9% accuracy on the semi-private evaluation set. The findings highlight the importance of candidate selection over mere generation, revealing that traditional methods like self-consistency can hinder performance by reducing hypothesis diversity.
A new approach to reasoning in LLMs reveals that holistic judging can significantly outperform traditional methods, achieving a record 72.9% accuracy on ARC-AGI-2.
Large language models can produce fluent, internally coherent reasoning traces for abstract reasoning tasks while still being confidently wrong - making selection among candidates, not just generation, the central challenge. I present a solver for ARC-AGI-2, a few-shot visual reasoning benchmark, built around two principles: (i) treating reasoning modalities as search operators, generating diverse candidates independently across text, image, and code channels, and (ii) context-preserving holistic judging, in which a judge model jointly compares all candidate reasoning traces within a single long-context prompt. Unlike self-consistency or majority voting, this approach reliably recovers correct minority hypotheses on tasks where the modal answer is wrong. On the ARC Prize semi-private evaluation set, the solver achieves 72.9 percent at USD 38.99 per task - the highest score on the verified leaderboard at the time of writing, exceeding the best standalone frontier models, GPT-5.2 Pro at 54.2 percent and Gemini 3 Pro at 54.0 percent, by +18.7 percentage points. On the public evaluation set, it achieves 76.1 percent at USD 19.69 per task. I release the full source code and document extensive negative results, including the finding that prescriptive prompting templates and iterative refinement systematically reduce hypothesis diversity and degrade performance.
