Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates methods for predicting the performance gain of Retrieval-Augmented Generation (RAG) in question answering. The authors evaluate pre-retrieval, post-retrieval, and post-generation predictors, including a novel post-generation predictor that models semantic relationships between the question, retrieved passages, and generated answer. Their findings demonstrate that this novel supervised predictor achieves the highest prediction quality for RAG performance.
Knowing whether RAG will actually *help* your QA system is now more predictable, thanks to a new method that smartly compares question, retrieved context, and answer.
We address the task of predicting the gain of using RAG (retrieval augmented generation) for question answering with respect to not using it. We study the performance of a few pre-retrieval and post-retrieval predictors originally devised for ad hoc retrieval. We also study a few post-generation predictors, one of which is novel to this study and posts the best prediction quality. Our results show that the most effective prediction approach is a novel supervised predictor that explicitly models the semantic relationships among the question, retrieved passages, and the generated answer.