Search papers, labs, and topics across Lattice.
Standard KV cache eviction fails on long-horizon reasoning models because decoding periodically generates "Thought Revisiting Tokens" that jump back to early planning context, violating the recency assumptions of existing methods. To resolve this, BeaconKV clusters query embeddings to identify recurring retrieval patterns and maintains compact "beacon queries" as global anchors to safeguard distant, high-utility KV pairs without retraining. Across four open-source reasoning models, the approach compresses the KV cache by up to 5.8脳 and boosts throughput by over 4.3脳 while preserving full-cache task accuracy.
Long-horizon reasoning traces repeatedly backtrack to early planning steps via predictable query clusters, revealing that a handful of representative "beacon" vectors can guide 5.8脳 KV cache compression with near-zero quality loss.
Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) cache grows linearly with sequence length and creates severe memory bottlenecks, often exceeding GPU capacity for long reasoning traces. Existing KV cache compression methods rely on recent queries to estimate future token importance, implicitly assuming these serve as reliable proxies for future attention patterns. We demonstrate that this assumption fails in long-horizon reasoning: certain decoding steps generate Thought Revisiting Tokens (TRT) that re-attend to distant previous context, such as task-solving plans formulated early in the trace. Through systematic analysis, we discover that queries corresponding to the TRT cluster into a small number of similarity groups in the embedding space. Based on this insight, we propose BeaconKV, a training-free KV cache compression method that maintains beacon queries, compact representatives for each global query cluster, to anticipate which KV pairs will be revisited without storing the entire query history. Across four open-source LRMs and diverse reasoning benchmarks, BeaconKV generally outperforms existing compression methods, achieving up to $5.8\times$ memory reduction while nearly preserving full cache accuracy and improving throughput by over $4.3\times$.
