Search papers, labs, and topics across Lattice.
This paper introduces STAIR, a novel retrieval system that leverages the global structure of documents, specifically utilizing Tables of Contents (ToC), to enhance the performance of Large Language Models (LLMs) in Retrieval Augmented Generation (RAG) tasks. By integrating ToC into a Differentiable Search Index (DSI), STAIR significantly reduces hallucinations to less than 0.05% while achieving a Recall@1 score of 82.6%, outperforming traditional methods like BM25 and DPR. The authors also release SearchTome, a benchmark dataset derived from 18 books across six domains, to facilitate further research in structure-aware retrieval techniques.
Harnessing the power of document structure, STAIR achieves a remarkable 82.6% Recall@1, setting a new standard for low-hallucination information retrieval in LLMs.
Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs are improving at handling long context, but still suffer from"lost in the middle"problem. Thus, precise and accurate retrieval is important. Current retrievers chunk long context into length-based manageable chunks - in the process throwing away rich and informative semantic global structure in the corpus. We introduce a novel retrieval system STAIR that empowers an LLM to exploit global structure in a corpus such as a Table of Contents (ToC) to efficiently store and retrieve information from its model parameters. Our thorough and careful ablation studies with a finetuned Differentiable Search Index (DSI) system show that ToC helps build a low hallucination (less than 0.05%) generative Information Retrieval (IR) system and can generalize to examples where very few training samples are available. To further research in this novel direction of ToC based retrieval we release SearchTome - a diverse benchmark created from 18 books across 6 diverse domains to further research in this novel direction. STAIR achieves a high Recall@1 score of 82.6% on SearchTome as compared to DSI (76.9%), where the difference is found to be statistically significant. STAIR easily beats other strong baselines such as BM25 (59.5%), DPR (68.7%) and out-of-the-box Mistral (13.8%).
