Search papers, labs, and topics across Lattice.
This paper introduces VisCache, a novel framework for efficient long-context inference in Vision Large Language Models (VLLMs) that addresses the high computational and memory costs associated with visual Key-Value (KV) caches. By implementing a two-stage pruning approach that filters redundant keyframes and employs a surgical KV compression algorithm (PruneKV), VisCache achieves significant improvements in inference speed and memory usage without compromising performance. Experimental results show that VisCache can deliver up to a 2.35× speedup and maintain competitive performance with only 19–28% of the original KV cache retained, setting a new standard for efficiency in VLLM inference.
Achieving up to 2.35× faster inference with only 19–28% of the original KV cache, VisCache redefines efficiency in Vision Large Language Models.
While Vision Large Language Models (VLLMs) have achieved remarkable success in multimodal reasoning, their long-context inference remains prohibitively expensive due to the massive computation and memory overhead of visual Key-Value (KV) caches. Existing KV compression methods often apply uniform pruning across visual tokens and layers, leading to substantial information loss and degraded performance.To address this challenge, we propose \textbf{VisCache}, a plug-and-play framework for coarse-to-fine \textbf{Vis}ual KV \textbf{Cache} pruning without training, which consists of two synergistic stages. First, a lightweight VLM filters temporal redundancy by selectively forwarding semantically informative keyframes. Second, we introduce {PruneKV}, a surgical KV compression algorithm tailored to the attention dynamics of VLLMs. Unlike rigid pruning strategies, PruneKV adopts a parabolic layer-wise budget allocation together with an asymmetric update mechanism that selectively prunes keys while fusing values, thereby preserving critical contextual information. Extensive experiments demonstrate that VisCache substantially improves inference efficiency, achieving up to {2.35$\times$ speedup} and significant memory reduction while maintaining competitive performance with only {19--28\%} KV cache retention. VisCache consistently outperforms existing baselines, establishing a new Pareto frontier between efficiency and performance for long-context VLLM inference. Code is available at https://github.com/Wlklk/VisCache
