Search papers, labs, and topics across Lattice.
The paper introduces RPDR, a data augmentation framework to improve dense retrievers for long-tail question answering by generating synthetic data, selecting high-quality easy-to-learn instances using a Round-Trip prediction method, and training the retriever with these instances. RPDR addresses the challenge of limited knowledge acquisition and recall in LLMs and dense retrieval models when dealing with rare or niche information. Experiments on PopQA and EntityQuestion datasets demonstrate that RPDR significantly outperforms existing retrievers, particularly in extremely long-tail categories.
Forget manually curating training data: RPDR automatically synthesizes and selects "easy-to-learn" examples, boosting dense retriever performance on long-tail QA tasks.
Long-tail question answering presents significant challenges for large language models (LLMs) due to their limited ability to acquire and accurately recall less common knowledge. Retrieval-augmented generation (RAG) systems have shown great promise in mitigating this limitation by integrating external retrieval mechanisms. However, dense retrieval models often face the same difficulties when generalizing to rare or niche knowledge. In this study, we introduce RPDR, a novel data augmentation framework that selects high-quality easy-to-learn training data, to enhance dense retrievers. Our approach is built around three core components: synthetic data generation, data selection with Round-Trip prediction to identify easy-to-learn instances, and retriever training with these instances. We evaluate RPDR on two long-tail retrieval benchmarks, PopQA and EntityQuestion, demonstrating substantial improvements over existing retrievers like BM25 and Contriver, especially on extremely long-tail categories. We identify the strengths and limitations of RPDR through detailed human analysis and propose a dynamic routing mechanism to dynamically route queries to specialized retrieval modules to further improve retrieval performance.
