Search papers, labs, and topics across Lattice.
This study investigates the relationship between final-answer accuracy and the preservation of reasoning in large models under KV cache compression, revealing a significant disconnect termed the "answer-evidence gap." By employing a controlled fixed-trace replay protocol, the authors demonstrate that while token-eviction methods can maintain competitive accuracy, they often fail to preserve the supporting rationales necessary for robust reasoning. The findings underscore the importance of evaluating not just the correctness of answers but also the integrity of the reasoning process that leads to those answers, particularly in critical applications like scientific QA and clinical calculations.
The study reveals that maintaining answer accuracy in large models can come at the cost of losing critical reasoning support, highlighting a significant "answer-evidence gap" in KV cache compression.
KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that supports it. We test this assumption for large reasoning models and show that it can fail: under compression, correct answers and the validity of their visible supporting rationales can be preserved at different rates. We study this failure with a controlled fixed-trace replay protocol, which holds reasoning content fixed and isolates whether compression preserves usable information from an already available trace. We evaluate ten token-eviction KV compression methods and one quantization method on three models across mathematical reasoning, scientific QA, clinical calculation, and long-context retrieval. We measure final accuracy, answer-chain consistency, and perturbation faithfulness. Across tasks, token-eviction methods can preserve competitive final-answer accuracy while substantially degrading chain support or perturbation faithfulness. We call this the answer-evidence gap. A coverage-preserving quantization control is substantially less affected, suggesting that the failure is tied less to KV memory reduction itself than to losing access to parts of the reasoning trace. Code is available at https://github.com/famous-blue-raincoat/Safe_KV_Compress.
