Search papers, labs, and topics across Lattice.
This paper introduces ReCo (Reward-Coordinated Compression), a novel framework that optimizes the inference efficiency of Large Reasoning Models (LRMs) by coordinating cache compression with a process reward signal. By recognizing that the tolerance for context loss varies along the reasoning trajectory, the method employs adaptive KV-cache compression, a penalty on redundant generation, and confidence-based early stopping to significantly reduce token generation. The results demonstrate a 37%-65% reduction in generated tokens and a 2.08x-2.35x decrease in end-to-end latency compared to traditional chain-of-thought methods, while maintaining accuracy levels.
Coordinating cache compression with a process reward can slash token generation by up to 65% without sacrificing accuracy.
Large Reasoning Models (LRMs) excel on complex tasks through long chain-of-thought (CoT) reasoning, but their lengthy intermediate steps cause severe overthinking that inflates inference cost. KV-cache compression is a common solution, yet existing reasoning-oriented methods apply a uniform policy across the trajectory and judge compression only by what it removes from the cache. Two observations point the other way. First, a reasoning state's tolerance to context loss varies along the trajectory, and process reward tracks it: deleting tokens at high-reward steps preserves accuracy far better than deleting the same budget at random. Second, compression is not free on the generation side, since a smaller cache leads the model to generate more tokens, partly canceling the saving. Together these motivate coordinating both sides under a single process reward. We propose ReCo (Reward-Coordinated Compression), a step-wise framework in which a lightweight process-reward estimator scores each completed step and drives three components: (1) reward-adaptive KV-cache compression that shrinks the retained cache harder at high-reward steps and less at low-reward ones, (2) a reward-banded penalty on reflection tokens that curbs redundant generation, and (3) confidence-based early stopping that triggers when the reasoning is reliable. Across three reasoning models and six benchmarks, ReCo reduces generated tokens by 37%-65% and end-to-end latency by 2.08x-2.35x over Full CoT, all while largely preserving accuracy.
