Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of a compact retriever that learns ranking behavior from score vectors, bypassing the need for access to teacher hidden states. By employing a row-centered score-vector objective and a memory-efficient PairMSE loss, the authors demonstrate that their distillation protocol can recover up to 50% of the performance gap between a large teacher model and a smaller student model. The resulting 0.6B student model significantly outperforms the teacher in terms of speed, achieving 4.7× faster query encoding and 9.7× faster document encoding, although external-transfer performance remains inconsistent.
Compact retrievers can achieve nearly half the performance of larger models while being up to 9.7× faster, challenging the assumption that bigger is always better in retrieval tasks.
Large embedding models improve retrieval quality, but serving large encoders online is expensive. We study whether a compact retriever can learn teacher ranking behavior from score vectors without access to teacher hidden states. The student trains on rows built from ground-truth positives and negative candidates produced by our data generation pipeline; we evaluate student-teacher hard-negative mining separately as an extension. We use a row-centered score-vector objective, a memory-efficient implementation of uniform all-pairs PairMSE loss. On a fixed eight-task evaluation panel, our distillation protocol recovers up to 50\% of the base-to-teacher gap. The distilled 0.6B student is 4.7$\times$ faster for query encoding and 9.7$\times$ faster for document encoding than sequential online teacher fusion. External-transfer performance after distillation remains mixed, so our evidence supports compression of teacher rankings under matched retrieval protocols.