Search papers, labs, and topics across Lattice.
This paper introduces ReLAT, a self-supervised test-time training method that enhances latent reasoning by reconstructing queries from latent states, effectively closing the loop on the reasoning process. By anchoring the latent computation to the original query, ReLAT ensures that the latent representation retains task-relevant information, which is crucial for accurate reasoning. The method demonstrates significant performance improvements across various benchmarks, achieving a 16.6-point accuracy increase on the AIME 2024 task compared to the best open-loop baseline.
Latent reasoning can now be validated in real-time, with a novel method that boosts accuracy by over 16 points on critical benchmarks.
Recent work moves intermediate reasoning from natural-language traces into latent or cache-level representations to reduce token overhead and avoid a discrete communication bottleneck. However, this shift also removes a key advantage of textual reasoning: intermediate states are no longer inspectable, making it difficult to determine whether a latent state still preserves the constraints of the original query. As a result, latent reasoning typically operates in an open loop, where a latent state is produced and consumed without an input-anchored fidelity check. We propose ReLAT (Reconstruction-Guided Latent Reasoning At Test Time), a self-supervised test-time training method that closes this loop using the query itself as the reference. Our key observation is that if a latent state faithfully represents a query, the query should be recoverable from it; if the query cannot be recovered, the latent state has lost task-relevant information. ReLAT operationalizes this principle by constructing a differentiable Question -> Latent Thought -> Question cycle and optimizing query reconstruction loss through the latent thought before answer generation. This anchors opaque latent computation to the problem specification it is supposed to represent. Across mathematical reasoning, knowledge QA, and code generation benchmarks on the Qwen family, ReLAT consistently improves over single-model inference, text-based collaboration, open-loop latent collaboration, and alternative test-time training objectives. On Qwen3-8B, ReLAT raises AIME 2024 accuracy from 56.7% to 73.3%, a 16.6-point gain over the strongest open-loop latent baseline.
