Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces a method for identifying whether an LLM's response is primarily based on information provided in the prompt (faithfulness) or on its internal knowledge (factuality). They create AttriWiki, a self-supervised data pipeline that automatically generates labeled examples by prompting models to either recall withheld entities or read them from context. They then train linear probes on model hidden representations using AttriWiki data, demonstrating that these probes can reliably predict the dominant knowledge source behind each output with high accuracy across various models and benchmarks.
LLMs often hallucinate, but a simple probe can reveal whether the error stems from misusing the prompt or from faulty internal knowledge, paving the way for targeted mitigation strategies.
Large language models (LLMs) often generate fluent but unfounded claims, or hallucinations, which fall into two types: (i) faithfulness violations - misusing user context - and (ii) factuality violations - errors from internal knowledge. Proper mitigation depends on knowing whether a model's answer is based on the prompt or its internal weights. This work focuses on the problem of contributive attribution: identifying the dominant knowledge source behind each output. We show that a probe, a simple linear classifier trained on model hidden representations, can reliably predict contributive attribution. For its training, we introduce AttriWiki, a self-supervised data pipeline that prompts models to recall withheld entities from memory or read them from context, generating labelled examples automatically. Probes trained on AttriWiki data reveal a strong attribution signal, achieving up to 0.96 Macro-F1 on Llama-3.1-8B, Mistral-7B, and Qwen-7B, transferring to out-of-domain benchmarks (SQuAD, WebQuestions) with 0.94-0.99 Macro-F1 without retraining. Attribution mismatches raise error rates by up to 70%, demonstrating a direct link between knowledge source confusion and unfaithful answers. Yet, models may still respond incorrectly even when attribution is correct, highlighting the need for broader detection frameworks.