Search papers, labs, and topics across Lattice.
This paper introduces HSRM, a hidden-state reward model that enhances test-time verification of mathematical reasoning by leveraging the internal representations of large language models (LLMs) instead of relying on expensive text-based verifiers. By extracting hidden states from a frozen generator at reasoning-step boundaries and employing a small Transformer encoder for candidate ranking, HSRM efficiently identifies correct solutions with significantly fewer parameters. The results demonstrate that HSRM matches or surpasses the performance of a larger text-only energy verifier across multiple benchmarks, highlighting its effectiveness and efficiency in verification tasks.
HSRM achieves competitive verification performance with just 2M parameters, challenging the notion that larger models are always necessary for accurate reasoning.
Large language models can often generate plausible mathematical reasoning traces, but reliably identifying the correct solution among multiple candidates remains a key challenge. Existing test-time reasoning pipelines typically rely on text-based verifiers that re-read each generated solution, making verification an expensive component of inference. Prior work has shown, however, that LLMs often encode correctness-related signals in their internal representations, including awareness of when their own answers are likely to be wrong. Building on this observation, we introduce HSRM, a lightweight hidden-state reward model that verifies candidate solutions by directly reading the generator's internal representations rather than re-processing its text. HSRM extracts hidden states from a frozen generator at reasoning-step boundaries and uses a small Transformer encoder to rank candidates. It is trained from self-generated trajectories with outcome labels, requiring neither human-written process supervision nor a large pretrained verifier. Across four mathematical reasoning benchmarks, HSRM matches or outperforms a 55M-parameter text-only energy verifier in 15 of 16 generator--dataset settings while using only about 2M parameters, providing an efficient alternative to text-only verification by reusing representations already computed during generation.
