Search papers, labs, and topics across Lattice.
LightSTAR introduces an efficient framework for visual document retrieval that significantly reduces computational costs by avoiding intensive Multi-modal Large Language Model (MLLM) encoding on every page. By leveraging keyword-anchored user queries, the method first employs LLM-free visual selection to generate a high-recall candidate set, followed by a vision-adaptive semantic refinement that enhances accuracy through region-wise feature fusion. Experimental results indicate that LightSTAR achieves state-of-the-art retrieval accuracy while dramatically decreasing end-to-end latency, addressing the critical accuracy-efficiency trade-off in the field.
LightSTAR slashes retrieval latency while maintaining top-tier accuracy by cleverly bypassing heavy MLLM processing on every document page.
Visual document retrieval requires rapidly locating relevant pages from large multi-modal corpora in response to user queries. While recent methods powered by Multi-modal Large Language Models (MLLMs) show competitive accuracy, they suffer from prohibitive computational costs by applying intensive MLLM encoding to every single page. Meanwhile, we observe that user queries are typically keyword-anchored, containing semantically rich words that are expected to appear directly in the visible text of relevant pages, offering an efficient cue for quickly narrowing down candidate pages. Building on this insight, we propose LightSTAR, an efficient framework that decomposes visual document retrieval into: 1) LLM-free Visual Selection, which utilizes content-grounded query encoding to focus on informative words and employs LLM-free visual embeddings to produce a high-recall candidate set; and 2) Vision-adaptive Semantic Refinement, which further performs fine-grained semantic matching exclusively on these top candidates via adaptive region-wise feature fusion to effectively combine textual and layout cues, optimized through a hardness-aware contrastive objective. Experimental results demonstrate that LightSTAR achieves state-of-the-art retrieval accuracy while reducing end-to-end latency by several-fold, offering a highly practical solution to the accuracy-efficiency trade-off in visual document retrieval. Code is available at https://github.com/bokufa/LightSTAR.
