Search papers, labs, and topics across Lattice.
This paper addresses the reproducibility gap in state-of-the-art retrieval models by introducing an open end-to-end training approach for multilingual retrieval, utilizing a novel translate-train method. The authors curate a substantial dataset of 665M English contrastive pairs and develop two models, DenseOn and LateOn, which achieve new state-of-the-art nDCG@10 scores of 56.20 and 57.22, respectively. Notably, while both models share architecture and training data, they exhibit distinct performance characteristics, with the late-interaction model demonstrating superior generalization to unseen languages.
Late-interaction models can outperform dense models in multilingual retrieval, revealing a new avenue for enhancing cross-lingual performance.
State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to multilingual retrieval through translate-train. We first reconstruct and curate 665M English contrastive pre-training pairs from 1.4B pairs across 34 public sources and build 1.88M supervised fine-tuning pairs with mined hard negatives. Training yields two 149M-parameter models: DenseOn, a single-vector dense model, and LateOn, a ColBERT-style late-interaction model. They achieve 56.20 and 57.22 average nDCG@10 on BEIR, respectively, setting new state-of-the-art results for this size class. We then translate the validated English data into eight languages, yielding 2.8B pairs with cross-lingual samples, and train mDenseOn and mLateOn, two 307M-parameter models built on mmBERT-base. Despite sharing their backbone, data, and objectives, their representations behave differently: the dense model is strong on English and translated languages but degrades outside translate-train support, whereas the late-interaction model generalizes better to unseen languages and scripts. This suggests that token-level matching turns translate-train from a target-language expansion strategy into a multilingual generalization recipe. We publicly release the models, datasets, and training code.
