Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces GRAB-RAG, a benchmark that evaluates the performance of small frozen retrieval-augmented generation (RAG) models under various context conditions, including misleading evidence. The findings reveal that while models effectively abstain from answering when evidence is missing, they still provide incorrect answers to 41.6% of misleading questions, often repeating the wrong information verbatim. Additionally, attempts to improve safety through conflict checks or NLI verifiers either reduce the model's coverage of correct answers or fail to address the misleading context issue effectively.
Models are misled by context, with 41.6% of answers to misleading questions echoing incorrect information, challenging the effectiveness of prompt-based abstention.
Missing and misleading evidence are not the same problem in retrieval-augmented generation (RAG), but prompt-based abstention treats them alike. Models abstain when context is absent, not when it is misleading. We introduce GRAB-RAG (Graded Abstention Benchmark for Retrieval-Augmented Generation), a paired benchmark that tests the same questions across four context conditions (supportive, degraded, missing, and misleading) in Natural Questions and HotpotQA. In the misleading condition, we edit a gold passage to support a wrong answer and place it among other retrieved passages. We test five abstention policies on three small frozen models (3.8B--8B) across two QA benchmarks. Models abstain reliably when evidence is missing, but under explicit abstention prompting still answer 41.6% of misleading questions, with 63% of those answers echoing the planted wrong entity verbatim. Chain-of-thought provides little additional benefit. A generator-side conflict check cuts the rate to 13.3% but discards many correct answers, while an NLI verifier recovers that coverage but fails when parametric memory and the misleading passage agree on the same wrong answer. Prompt-based abstention asks whether context is sufficient, not whether it is correct. Neither verifier closes this gap without trading coverage for safety.