Search papers, labs, and topics across Lattice.
This paper introduces ColBERTSaR, an innovative approach that applies product quantization to the ColBERT retrieval architecture, transforming its index into a true inverted index. By addressing the inefficiencies of gathering and decompressing document token embeddings, the authors achieve a significant reduction in index size—50-70% smaller than traditional one-bit PLAID indexes—while maintaining retrieval effectiveness. This advancement not only enhances scalability but also optimizes the retrieval process for ad hoc queries, making it more efficient for practical applications.
ColBERTSaR shrinks the index size by up to 70% without sacrificing retrieval performance, revolutionizing the efficiency of neural retrieval systems.
While ColBERT is an effective neural retrieval architecture, it requires a heavy index structure to support candidate set retrieval based on approximated token embeddings, gathering and decompressing document token embeddings, and applying the MaxSim operation. Indexes in PLAID and similar ColBERT implementations require five to ten times the disk storage of the original raw text, which limits their scalability. Furthermore, prior work has identified that the gathering and decompression stages are the primary inefficiencies at query time. Limiting the number of document tokens that must be gathered by thresholding and score approximation does not eliminate the need for the entire index to support ad hoc queries. In this work, we propose an embedding quantization approach that turns a ColBERT index into a true inverted index. We show that, theoretically, ColBERT with embedding quantization is equivalent to learned-sparse retrieval except for the scoring mechanism. Empirically, we demonstrate that our index is 50-70% smaller than a one-bit PLAID index while retaining retrieval effectiveness.
