Search papers, labs, and topics across Lattice.
This paper introduces ReLAR, a reinforcement-guided latent refinement framework designed to enhance the stability of reasoning in large language models (LLMs) by iteratively refining hidden states before decoding. By employing learned depth and action controllers, ReLAR adaptively determines the number and direction of refinement steps based on a policy gradient objective that focuses on likelihood improvement. Experimental results across various reasoning benchmarks demonstrate that ReLAR significantly boosts accuracy and generation quality while reducing inference overhead compared to traditional explicit reasoning methods.
ReLAR achieves improved reasoning stability and accuracy in LLMs with significantly lower inference costs, challenging the notion that complex reasoning always requires elaborate chain-of-thought processes.
Large language models show strong reasoning ability, but their internal reasoning process can remain unstable in complex multi-step settings, where early hidden-state errors may propagate to incorrect predictions. We propose ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations before decoding. ReLAR maintains a compact latent reasoning state and uses learned depth and action controllers to adaptively determine both the number and direction of refinement steps. The controllers are trained with a policy gradient objective based on step-wise likelihood improvement, enabling efficient input-dependent reasoning without explicit chain-of-thought generation. Experiments on medical, mathematical, multi-hop reasoning, and open-ended generation benchmarks show that ReLAR improves accuracy, generation quality, and reasoning stability with substantially lower inference overhead than explicit reasoning baselines.
