Search papers, labs, and topics across Lattice.
The paper introduces Sali-Cache, a proactive KV-cache optimization framework for VLMs processing long-form video, addressing the memory bottleneck caused by the linear scaling of the KV-cache with sequence length. Sali-Cache employs a dual-signal approach, using optical flow to detect temporal redundancy and saliency detection to identify spatially significant regions, enabling intelligent memory allocation before attention computation. Experiments on LLaVA 1.6 demonstrate a 2.20x compression ratio in effective memory usage while maintaining 100% accuracy across BLEU, ROUGE-L, and Exact Match metrics, allowing for efficient long-form video processing.
Get 2.2x memory savings in your video VLMs without sacrificing accuracy by proactively pruning the KV-cache using optical flow and saliency, *before* computing attention.
Vision-Language Models (VLMs) face a critical memory bottleneck when processing long-form video content due to the linear growth of the Key-Value (KV) cache with sequence length. Existing solutions predominantly employ reactive eviction strategies that compute full attention matrices before discarding tokens, resulting in substantial computational waste. We propose Sali-Cache, a novel a priori optimization framework that implements dual-signal adaptive caching through proactive memory management. By integrating a temporal filter based on optical flow analysis for detecting inter-frame redundancy and a spatial filter leveraging saliency detection for identifying visually significant regions, Sali-Cache intelligently manages memory allocation before entering computationally expensive attention operations. Experimental evaluation on the LLaVA 1.6 architecture demonstrates that our method achieves a 2.20x compression ratio in effective memory usage while maintaining 100% accuracy across BLEU, ROUGE-L, and Exact Match metrics. Furthermore, under identical memory budget constraints, Sali-Cache preserves context-rich features over extended temporal durations without degrading model performance, enabling efficient processing of long-form video content on consumer-grade hardware.
