Search papers, labs, and topics across Lattice.
This paper introduces AnchorFold, a training-free framework that enhances multi-vector visual document retrieval (VDR) by employing Recursive Attention Propagation to efficiently compress document-side indices. The method focuses on selecting high-centrality tokens as anchors and aggregates remaining tokens based on their similarity to these anchors, significantly reducing overhead while preserving retrieval quality. Experimental results demonstrate that AnchorFold achieves near-lossless compression, retaining 98.3% of full-index NDCG@5 at 5x compression and 92.4% at 20x compression across multiple retrieval backbones.
Achieving near-lossless compression in visual document retrieval, AnchorFold retains over 98% retrieval quality even at 5x compression ratios.
Multi-vector vision-language retrievers enable fine-grained Visual Document Retrieval (VDR) through late interaction, but storing and scoring hundreds of visual patch embeddings per page incurs substantial overhead. Existing training-free methods rely on pruning or merging: pruning degrades sharply under aggressive compression, whereas merging does not explicitly prioritize important regions when forming representatives. We introduce AnchorFold, a training-free focus-then-fold framework for document-side index compression. AnchorFold applies Recursive Attention Propagation over visual self-attention graphs, performing multi-step propagation within each attention head and integrating scores across heads and layers. The focus stage selects the highest-centrality tokens as anchors. The fold stage assigns remaining tokens to their most similar anchors in the normalized retrieval space and summarizes each anchor-centered group through centrality-weighted aggregation. This preserves non-anchor contributions while concentrating capacity on structurally important tokens. Across ViDoRe v1/v2 and REAL-MM-RAG with three diverse retrieval backbones, AnchorFold consistently outperforms all evaluated training-free baselines at $纬\leq 0.20$. On ViDoRe v1/v2, it retains 98.3% of full-index NDCG@5 on average at $5\times$ compression, achieving near-lossless compression, and 92.4% at $20\times$ compression.
