Search papers, labs, and topics across Lattice.
To address the failure of current serving systems to reuse KV caches in non-prefix settings like dynamic RAG chunk assembly and multi-agent coordination, the authors establish KVShareArena to systematically benchmark cache-repair and transfer methods across arbitrary context positions and checkpoints. Standardizing this space is critical because naive reuse injects corrupted position encodings and ignores cross-source attention, often performing worse than generating from scratch. Benchmarking across multiple model families reveals that while zero-compute position correction handles single-source tasks, multi-source reasoning strictly requires partial re-encoding or trained adapters to recover 50–66% of the quality gap between no cache and full recomputation.
Unrepaired KV caches can perform worse than no cache at all, showing that zero-overhead position shifts break down the moment a query demands cross-attention between multiple sources.
LLM serving systems already reuse KV caches, but only when the reused text sits at the very start of the prompt. Two growing workloads break this condition: a retrieval-augmented generation server assembles a different set of retrieved chunks for every query, and a multi-agent coordinator reads reports written by other agents. Reused inside a new prompt, a cache carries the wrong positions and never attended to the other sources. The cache may also have been written by a different checkpoint of the same model family, which changes the stored values. Repair methods for such caches have appeared in three separate communities, each measured on its own terms, and existing benchmarks test only exact-prefix reuse, where nothing is lost. KVShareArena benchmarks KV-cache reuse across prompt contexts and model checkpoints on retrieved chunks and agent reports. It scores every method by the fraction of the gap it recovers between no cache and full recomputation, and charges compute, memory, and per-request latency with the cache in hand, reporting the one-time cost of building a cache separately. We find that correcting positions, which needs no recomputation, is enough until a question needs several sources at once. There, only methods that pay, by re-encoding part of the cache or by training, recover half to two thirds of the gap; unrepaired caches can be worse than no cache. Cache-compression methods that are harmless on a single prompt fall significantly behind position correction on freshly written agent reports. These patterns hold across three model boards. When a different checkpoint wrote the cache, training-free methods are barely affected, while an adapter trained on one checkpoint's caches loses quality. Harness, frozen querysets, and cost accounting ship as a pip package with an automated submission workflow and a public leaderboard.
