Search papers, labs, and topics across Lattice.
This study investigates the concept of surrogate fidelity in mechanistic interpretability by assessing how well measurements from open language models can inform understanding of closed models. The authors evaluate prediction, attribution, and representation levels across eleven models from four families, revealing that while prediction fidelity is high, it often misrepresents attribution fidelity, leading to an access-validity inversion. The findings underscore that agreement in predictions does not guarantee similar causal explanations, challenging assumptions about model interpretability across different architectures.
Prediction accuracy in language models can be misleading, as it often obscures significant differences in causal attributions.
Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log-probabilities over output tokens. This creates a surrogate problem: when do measurements made on open models allow us to make claims about a closed model? We evaluate surrogate fidelity at the prediction, attribution, and representation levels. For binary classification tasks, log-odds provide an API-compatible scalar readout of the model's representation space, and leave-one-out attributions provide insight into model behavior. Across eleven models spanning four families (Llama, Qwen, GPT, and Gemini), we find that prediction fidelity substantially overstates attribution fidelity: models that agree on what the answer is often disagree on why. We document an access-validity inversion: white-box signals like attention patterns and perturbation magnitudes are highly stable across models but only weakly predictive of causal attributions, which black-box input ablations capture by design. Mechanistic insight does not automatically transfer to closed targets, and prediction-level agreement is insufficient to warrant such transfer. Code and results are available at https://github.com/facebookresearch/surrogate.
