Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a Causal Graph Attention Network (GCAN) to mitigate hallucinations in LLMs by interpreting internal attention flows. GCAN constructs token-level graphs using self-attention weights and gradient-based influence scores, quantifying factual dependency with a Causal Contribution Score (CCS). A fact-anchored graph reweighting layer then reduces the influence of hallucination-prone nodes, leading to a 27.8% reduction in hallucination rate and a 16.4% improvement in factual accuracy on TruthfulQA and HotpotQA compared to RAG baselines.
LLMs can be made significantly more truthful by identifying and down-weighting the influence of hallucination-prone tokens during generation.
This paper primarily focuses on the hallucinations caused due to AI language models(LLMs).LLMs have shown extraordinary Language understanding and generation capabilities .Still it has major a disadvantage hallucinations which give outputs which are factually incorrect ,misleading or unsupported by input data . These hallucinations cause serious problems in scenarios like medical diagnosis or legal reasoning.Through this work,we propose causal graph attention network (GCAN) framework that reduces hallucinations through interpretation of internal attention flow within a transformer architecture with the help of constructing token level graphs that combine self attention weights and gradient based influence scores.our method quantifies each tokens factual dependency using a new metric called the Causal Contribution Score (CCS). We further introduce a fact-anchored graph reweighting layer that dynamically reduces the influence of hallucination prone nodes during generation. Experiments on standard benchmarks such as TruthfulQA and HotpotQA show a 27.8 percent reduction in hallucination rate and 16.4 percent improvement in factual accuracy over baseline retrieval-augmented generation (RAG) models. This work contributes to the interpretability,robustness, and factual reliability of future LLM architectures.