Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To determine whether token-level and latent intermediate computations share identical algorithmic mechanisms, the authors trained five GPTNeoX variants from scratch on a multi-hop graph reasoning task (ProsQA-Ext), spanning vanilla, Chain-of-Thought (CoT), Pause Token, and end-to-end latent reasoning architectures. While all variants achieve high in-distribution accuracy, vanilla, CoT, and Pause models rely on brittle local graph heuristics that collapse under longer reasoning hops, whereas latent models achieve robust depth generalization. Mechanistic circuit analysis and causal interventions reveal that this latent generalization is driven by a sparsely localized recurrent forward-search algorithm that updates reachability states across latent steps.
While Chain-of-Thought models overfit to local heuristics and collapse on longer reasoning chains, latent reasoning architectures autonomously discover a sparse, recurrent forward-search circuit that generalizes to unseen depths.
Large Language Models can perform multi-step reasoning and improve task performance through different forms of intermediate computation, from token-based traces to computation carried out in latent space. However, a question remains open: do these different forms of thinking rely on the same underlying mechanism? To address this, we train and compare five variants of the same GPTNeoX backbone from scratch on an extended multi-hop reasoning task (ProsQA-Ext): a vanilla model, a Chain-of-Thought (CoT) model, a Pause Token model, and two latent-reasoning models that are optimized end-to-end without intermediate reasoning traces. We find that, strong in-distribution (ID) performance does not guarantee depth generalization. Vanilla, CoT, and Pause Token models solve ID problems well, but rely largely on local graph features and generalize poorly to out-of-distribution (OOD) problems with longer hops. In contrast, latent variants generalize better and show internal dynamics consistent with forward reachability propagation on the graph. Causal interventions and circuit analysis localize this computation to a sparse recurrent search circuit in the bottleneck latent model: an attention head retrieves graph relations, an MLP and the residual stream update the reachability state across recurrent steps, while multiple attention heads together then do the candidate matching. Together, these results show that different thinking mechanisms can learn distinct computational solutions, even at similar ID performance. In this setting, latent recurrence supports a reusable forward-search algorithm that generalizes beyond the training depth.