Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ViSAR, a training-free adaptive-$k$ retrieval method for Visual Document Question Answering (DocVQA) that dynamically adjusts the number of retrieved document pages based on query complexity. By constructing a query-conditioned page-level similarity matrix in the embedding space, ViSAR significantly reduces retrieval latency by up to 58.7% while maintaining or enhancing answer accuracy compared to traditional fixed top-$k$ methods. The findings indicate a strong correlation between the similarity matrix structure and answer accuracy, paving the way for improved retrieval quality in document understanding tasks.
Adaptive retrieval can cut RAG latency by nearly 59% without sacrificing accuracy, revolutionizing how we approach document question answering.
Document Visual Question Answering (DocVQA) often leverages Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large Vision-Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k$ number of pages regardless of query complexity, which increases LVLM latency and may degrade answer accuracy. We introduce ViSAR (Visual Semantic Activation Retrieval), a training-free adaptive-$k$ retrieval method for late-interaction visual document retrieval. ViSAR operates directly in the embedding space to construct a query-conditioned page-level similarity matrix that highlights query-relevant semantics and dynamically determines the number of pages to retrieve. Across multiple encoders and LVLMs, ViSAR retrieves compact, query-adapted page sets that reduce RAG latency by up to 58.7\%, while maintaining or improving answer accuracy compared with fixed top-$k$ and adaptive retrieval heuristics. Furthermore, we show that the similarity matrix structure correlates with answer accuracy, suggesting future directions for retrieval quality-aware document understanding.