Search papers, labs, and topics across Lattice.
The paper introduces MIGRASCOPE, a novel benchmark for RAG retrievers grounded in information theory, enabling the quantification of retrieval quality, redundancy, synergy, and marginal contribution. They apply MIGRASCOPE to analyze state-of-the-art retrievers across major RAG corpora, revealing insights into their individual strengths and weaknesses. The study demonstrates that carefully chosen ensembles of retrievers outperform any single retriever, offering practical guidance for building more effective RAG systems.
Stop guessing which RAG retriever to use: this benchmark uses information theory to reveal the hidden strengths, weaknesses, and synergies between them.
Retrieval-Augmented Generation (RAG) systems rely critically on the retriever module to surface relevant context for large language models. Although numerous retrievers have recently been proposed, each built on different ranking principles such as lexical matching, dense embeddings, or graph citations, there remains a lack of systematic understanding of how these mechanisms differ and overlap. Existing benchmarks primarily compare entire RAG pipelines or introduce new datasets, providing little guidance on selecting or combining retrievers themselves. Those that do compare retrievers directly use a limited set of evaluation tools which fail to capture complementary and overlapping strengths. This work presents MIGRASCOPE, a Mutual Information based RAG Retriever Analysis Scope. We revisit state-of-the-art retrievers and introduce principled metrics grounded in information and statistical estimation theory to quantify retrieval quality, redundancy, synergy, and marginal contribution. We further show that if chosen carefully, an ensemble of retrievers outperforms any single retriever. We leverage the developed tools over major RAG corpora to provide unique insights on contribution levels of the state-of-the-art retrievers. Our findings provide a fresh perspective on the structure of modern retrieval techniques and actionable guidance for designing robust and efficient RAG systems.
