Search papers, labs, and topics across Lattice.
This paper introduces DistilVDR, a compact end-to-end visual document retrieval system distilled from a larger vision-language model, achieving significant efficiency gains. By employing a dual-student distillation approach with a focus on a single-vector retriever, the model reduces index size and retrieval time while maintaining high performance. The resulting DistilVDR-HiRes variant achieves 61.74 average NDCG@5, outperforming all sub-1B baselines on high-resolution benchmarks and demonstrating a 15.6 times smaller index size compared to the strongest multi-vector baseline.
A single-vector visual document retrieval system that is 15.6 times smaller and an order of magnitude faster than existing multi-vector models while retaining high accuracy.
Visual document retrieval (VDR) is dominated by multi-billion-parameter models that are slow to index at full corpus scale and expensive to serve. Prior compression routes either train a smaller multi-vector encoder from scratch or distil only the query side; neither yields a compact single-vector retriever end-to-end. We present DistilVDR, a 524M end-to-end VDR system distilled bilaterally from a single 8B vision-language teacher under a pointwise cosine alignment loss. All supervision comes from the frozen teacher's embedding space, which was itself trained with relevance supervision, so the student objective needs no relevance labels, negative sampling, or contrastive term. We match VDR's text-query and image-document input asymmetry with an asymmetric encoder-only student that concentrates visual capacity on the document side and keeps the query side at 70M parameters. We release two variants that share the same encoders and training and differ only in the document encoder's visual-tile budget: DistilVDR-HiRes attains 61.74 average NDCG@5 on ViDoRe v1+v2+v3 (86.9% of the 8B teacher) and leads every reproduced sub-1B baseline on the high-resolution-sensitive v3 benchmark, while DistilVDR-Fast attains 59.98 at a 3 times smaller visual-token budget. Both variants store one million documents in a 15.6 times smaller index than the strongest sub-1B multi-vector baseline and index the corpus an order of magnitude faster. The code is available at https://github.com/Ryenhails/NanoVDR.
