Search papers, labs, and topics across Lattice.
This paper introduces QV-PIC, a novel query-aware dual-resolution caching framework that enhances the efficiency of Retrieval-Augmented Generation (RAG) by addressing the limitations of Position-Independent Caching (PIC) when using visual representations. By compiling visual caches under model-native templates and leveraging cumulative query relevance scores, QV-PIC significantly improves the quality of cached outputs while maintaining low computational overhead. Experimental results demonstrate that QV-PIC achieves a 21.6-point increase in average F1 score compared to standard rendered-image PIC, while also reducing total time to first token (TTFT) by 83.8%.
QV-PIC closes the quality gap between visual and text caching in RAG, achieving unprecedented efficiency gains without sacrificing performance.
Retrieval-Augmented Generation (RAG) repeatedly prefills identical text chunks across queries, incurring redundant computations. Position-Independent Caching (PIC) mitigates it by reusing precomputed Key-Value (KV) across positions, but its efficiency is constrained by the large volume of text tokens. Rendering text chunks as images can compress the text into fewer visual tokens, but the rendered-image PIC suffers more severe quality degradation than the text PIC. This representation-specific gap primarily arises from contextual mismatches across independently compiled caches and the loss of fine-grained textual evidence during visual compression. Existing PIC repair methods mainly address the former through selective recomputation, but they incur online computation and cannot recover lost textual details. We propose QV-PIC, a query-aware dual-resolution PIC reuse framework guided by model-native templates. Offline, QV-PIC compiles visual caches under the model's native chat-template prefix, improving PIC quality without online recomputation. Online, it preserves global context with low resolution and restores fine-grained textual evidence within a high-resolution budget by cumulative query relevance scores, retaining the efficiency benefit of visual compression. Across six tasks, QV-PIC improves average F1 by 21.6 points over vanilla rendered-image PIC, closes the gap to vanilla text PIC, and surpasses optimized text PIC by 2.58 F1 while reducing TTFT by 17.2\%. Relative to full prefill, it cuts TTFT by 83.8%.
