Search papers, labs, and topics across Lattice.
This paper introduces Selective Regenerative Decoding (SRD), a novel approach that enhances inference-time reasoning in large language models (LLMs) by allowing for selective refinement of candidate trajectories. Unlike traditional methods that treat trajectories as atomic units, SRD intelligently preserves high-quality prefixes while refining or discarding only the degraded suffixes, leading to significant improvements in sample efficiency. The method demonstrates a 1.28-to-1.36-fold gain in efficiency and matches the accuracy of Best-of-N strategies with fewer generated tokens across various benchmarks, highlighting its effectiveness in low-compute scenarios.
SRD redefines trajectory selection by enabling segment-level intervention, achieving better reasoning efficiency without the need for larger models.
Inference-time decoding methods improve LLM reasoning by exploring multiple candidate trajectories, yet treat each trajectory as atomic: either retaining it whole or discarding it irreversibly. This wastes computation on partially promising candidates whose high-quality prefixes are abandoned alongside degraded suffixes. We introduce Selective Regenerative Decoding (SRD), which routes each candidate to discard, keep, or refine only the degraded portion of the suffix while preserving the useful prefix of borderline candidates, without requiring a larger target model. Under mild assumptions, SRD achieves a provable 1.28-to-1.36-fold gain in sample efficiency over rejection sampling with strictly higher expected trajectory quality, with the gain growing as the candidate pool grows. Across MATH500, GPQA Diamond, HotpotQA, and AlpacaEval with multiple generation-reward model pairs, SRD matches Best-of-N accuracy with substantially fewer generated tokens and outperforms speculative rejection in low-compute regimes. By enabling segment-level intervention rather than whole-trajectory selection, SRD opens a previously underexplored region of the accuracy-compute tradeoff for inference-time reasoning.
Amazon Science