Search papers, labs, and topics across Lattice.
This study introduces a framework to evaluate the robustness of large language models (LLMs) against intrinsic hallucinations by employing semantically equivalent adversarial attacks. The researchers found that even state-of-the-art models exhibit significant degradation in contextual faithfulness, with a notable 50% reduction in accuracy for GPT-5-mini when subjected to these attacks. This highlights the fragility of LLMs' reliance on in-context evidence, underscoring the need for improved architectural and training strategies to ensure robust grounding.
Even state-of-the-art LLMs can lose up to 50% of contextual faithfulness when faced with semantically equivalent adversarial queries.
Large language models (LLMs) are often used in conjunction with external knowledge sources to improve their factual accuracy and decrease hallucinations, through methods such as Retrieval-Augmented Generation (RAG). However, these systems remain susceptible to intrinsic hallucinations, where the model generates unfaithful or fabricated information that is not supported by the retrieved evidence. We propose a novel framework to assess model robustness against this phenomenon by stress-testing using natural, semantically equivalent variations of a user query found via adversarial optimization methods. We apply our framework, which enforces strict semantic equivalence constraints and an intrinsic hallucination objective, to a range of adversarial attack techniques across white-box, gray-box, and black-box adversarial settings. Evaluating these attacks on 5 open-source and 5 closed-source generator models across 3 datasets, we demonstrate that even state-of-the-art models are highly susceptible to meaning-preserving perturbations, which significantly degrade contextual faithfulness (by up to 50% for GPT-5-mini). Our findings indicate that faithful use of in-context evidence remains fragile even in state-of-the-art LLMs, motivating architectures and training objectives that enforce robust grounding independent of surface query form. Code is available at: https://github.com/atriviveksharma/intrinsic_hall
