Search papers, labs, and topics across Lattice.
The paper introduces ConflictQA, a benchmark designed to evaluate LLMs' ability to reason faithfully when faced with conflicting information from text and knowledge graphs. Experiments reveal that LLMs struggle to identify reliable evidence in cross-source conflicts, exhibiting sensitivity to prompting and a tendency to rely exclusively on one source. To mitigate this, the authors propose XoT, a two-stage explanation-based framework, which demonstrates improved reasoning over conflicting evidence.
LLMs often fail to reconcile conflicting information from text and knowledge graphs, instead latching onto a single source based on prompting, highlighting a critical vulnerability in RAG systems.
Large language models (LLMs) have achieved remarkable success across a wide range of applications especially when augmented by external knowledge through retrieval-augmented generation (RAG). Despite their widespread adoption, recent studies have shown that LLMs often struggle to perform faithful reasoning when conflicting knowledge is retrieved. However, existing work primarily focuses on conflicts between external knowledge and the parametric knowledge of LLMs, leaving conflicts across external knowledge largely unexplored. Meanwhile, modern RAG systems increasingly emphasize the integration of unstructured text and (semi-)structured data like knowledge graphs (KGs) to improve knowledge completeness and reasoning faithfulness. To address this gap, we introduce ConflictQA, a novel benchmark that systematically instantiates conflicts between textual evidence and KG evidence. Extensive evaluations across representative LLMs reveal that, facing such cross-source conflicts, LLMs often fail to identify reliable evidence for correct reasoning. Instead, LLMs become more sensitive to prompting choices and tend to rely exclusively on either KG or textual evidence, resulting in incorrect responses. Based on these findings, we further propose XoT, a two-stage explanation-based thinking framework tailored for reasoning over heterogeneous conflicting evidence, and verify its effectiveness with extensive experiments.
NUS