Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the performance limitations of recursive reasoning models, which use repeated application of a small Transformer block for algorithmic tasks, by unifying their architectural and optimization differences under a controlled experimental framework. The study identifies a surprisingly simple recipe for stable and generalizable recursive reasoning, comprising an intermediate gradient horizon, large physical batches, and controlled recurrent state updates, without requiring explicit hierarchical architectures. A resulting 13.6M-parameter model demonstrates state-of-the-art performance across six algorithmic domains, significantly improving out-of-distribution generalization, particularly on Arithmetic and ARC-AGI-1.
Recursive reasoning models achieve state-of-the-art algorithmic performance and OOD generalization with a simple optimization recipe, decoupling architectural complexity from recurrent state stabilization.
Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic tasks. Such compact solvers are natural candidates for tools that an LLM can call on narrow algorithmic subproblems. However, existing models such as HRM, TRM and URM differ in architecture, gradient propagation and training procedure simultaneously. This makes it hard to tell what drives their performance, and their optimization is still poorly understood and often unstable. In this work we address both of these gaps. First, we study these questions under a unified experimental pipeline spanning six algorithmic domains. Individual controlled ablations are performed on representative domains, while the resulting recipe is evaluated across the full suite. The study reveals a surprisingly simple recipe for stable and generalizable recursive reasoning: an intermediate gradient horizon, large physical batches and controlled updates of the recurrent state. An explicit hierarchical architecture is not needed. Second, we combine these findings into a stable 13.6M-parameter model that achieves the strongest overall performance among the evaluated recursive baselines, with particularly large gains on out-of-distribution generalization. It raises Arithmetic OOD accuracy to 71.2%, from 36.2% for the strongest baseline, while reaching 98.41% on Sudoku and 59.5% pass@2 on ARC-AGI-1. Our results show that, within the recursive architectures studied here, performance depends strongly on how recurrence is optimized and stabilized. More broadly, it shows how AI systems can be improved by optimizing their components one at a time.