Search papers, labs, and topics across Lattice.
This paper addresses the limitations of traditional RAG systems that optimize for single-query retrieval, revealing that they only meet 41% of session-level information needs for enterprise users. By reorganizing the knowledge base using co-occurrence-aware clustering and enhancing retrieval through cluster neighborhoods, the authors achieve a significant increase in session coverage to 58% and reduce the number of retrieval calls needed. The findings suggest a paradigm shift in evaluation metrics for enterprise RAG systems, emphasizing session-level coverage over single-query recall.
A single retrieval call only meets 41% of user session needs, but with innovative knowledge base reorganization, we boost coverage to 58% while slashing retrieval calls by a third.
RAG systems retrieve documents optimized for answering one query at a time. Yet enterprise users arrive with sessions, that is, coherent episodes of related questions that span semantically distant parts of the knowledge base. We show that a single retrieval call over a standard knowledge base covers only 41% of a user's session-level information need. To close this gap, we reorganize the KB offline using co-occurrence-aware clustering and expand retrieval candidates through cluster neighborhoods at query time. On WixQA (6,221 enterprise support articles), our method raises single-query session coverage to 58% (+17% absolute; 95% CI: [14.1, 20.4]), reduces retrieval calls to 70% coverage by 34%, and compresses the KB to 20% of its original size, all consistently across four embedding models and six functional domains. We argue that session-level coverage, not single-query recall, should be the primary metric for enterprise RAG evaluation.
