Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces CUE-R, a framework for evaluating the utility of individual retrieved documents in Retrieval-Augmented Generation (RAG) systems. CUE-R uses intervention-based methods (REMOVE, REPLACE, DUPLICATE) on retrieved evidence and measures the impact on correctness, grounding faithfulness, confidence error, and trace divergence. Experiments on HotpotQA and 2WikiMultihopQA reveal that removing or replacing evidence harms performance, while duplication is often redundant but not behaviorally neutral, highlighting the limitations of answer-only evaluation.
Current RAG evaluation misses crucial evidence effects: CUE-R reveals that removing or replacing retrieved documents significantly harms correctness and grounding, while simply looking at the final answer masks these important dynamics.
As language models shift from single-shot answer generation toward multi-step reasoning that retrieves and consumes evidence mid-inference, evaluating the role of individual retrieved items becomes more important. Existing RAG evaluation typically targets final-answer quality, citation faithfulness, or answer-level attribution, but none of these directly targets the intervention-based, per-evidence-item utility view we study here. We introduce CUE-R, a lightweight intervention-based framework for measuring per-evidence-item operational utility in single-shot RAG using shallow observable retrieval-use traces. CUE-R perturbs individual evidence items via REMOVE, REPLACE, and DUPLICATE operators, then measures changes along three utility axes (correctness, proxy-based grounding faithfulness, and confidence error) plus a trace-divergence signal. We also outline an operational evidence-role taxonomy for interpreting intervention outcomes. Experiments on HotpotQA and 2WikiMultihopQA with Qwen-3 8B and GPT-5.2 reveal a consistent pattern: REMOVE and REPLACE substantially harm correctness and grounding while producing large trace shifts, whereas DUPLICATE is often answer-redundant yet not fully behaviorally neutral. A zero-retrieval control confirms that these effects arise from degradation of meaningful retrieval. A two-support ablation further shows that multi-hop evidence items can interact non-additively: removing both supports harms performance far more than either single removal. Our results suggest that answer-only evaluation misses important evidence effects and that intervention-based utility analysis is a practical complement for RAG evaluation.