Search papers, labs, and topics across Lattice.
This paper argues that interpretability research on LLMs often suffers from a lack of generalizability and overreaching causal interpretations, and proposes that causal inference, specifically Pearl's causal hierarchy, provides a framework for validating interpretability claims. The authors demonstrate how causal representation learning (CRL) can operationalize this hierarchy by specifying recoverable variables and necessary assumptions. They advocate for a diagnostic framework that aligns interpretability methods and evaluations with the strength of evidence to improve the generalizability of findings.
Interpretability studies on LLMs often overreach, but causal inference offers a framework to ensure claims about model behavior are actually valid and generalizable.
Interpretability research on large language models (LLMs) has yielded important insights into model behaviour, yet recurring pitfalls persist: findings that do not generalise, and causal interpretations that outrun the evidence. Our position is that causal inference specifies what constitutes a valid mapping from model activations to invariant high-level structures, the data or assumptions needed to achieve it, and the inferences it can support. Specifically, Pearl's causal hierarchy clarifies what an interpretability study can justify. Observations establish associations between model behaviour and internal components. Interventions (e.g., ablations or activation patching) support claims how these edits affect a behavioural metric (\eg, average change in token probabilities) over a set of prompts. However, counterfactual claims -- i.e., asking what the model output would have been for the same prompt under an unobserved intervention -- remain largely unverifiable without controlled supervision. We show how causal representation learning (CRL) operationalises this hierarchy, specifying which variables are recoverable from activations and under what assumptions. Together, these motivate a diagnostic framework that helps practitioners select methods and evaluations matching claims to evidence such that findings generalise.
