Search papers, labs, and topics across Lattice.
This paper introduces a scalable attribution patching framework to understand how individual latent components in CLIP models contribute to predictions, addressing the gap between identifying what components encode and how they influence outcomes. The framework combines instance-wise component attributions with semantic alignment scores to reveal unexpected semantic reliance and spurious correlations within CLIP's latent space. Experiments across multiple CLIP variants uncover hundreds of surprising components linked to polysemous words, visual typography, and dataset artifacts, demonstrating the method's ability to identify hidden biases and limitations.
CLIP models exhibit surprising reliance on latent components encoding polysemous words, visual typography, and dataset artifacts, revealing hidden biases that can be amplified in downstream tasks.
Transformer-based CLIP models are widely used for text-image probing and feature extraction, making it relevant to understand the internal mechanisms behind their predictions. While recent works show that Sparse Autoencoders (SAEs) yield interpretable latent components, they focus on what these encode and miss how they drive predictions. We introduce a scalable framework that reveals what latent components activate for, how they align with expected semantics, and how important they are to predictions. To achieve this, we adapt attribution patching for instance-wise component attributions in CLIP and highlight key faithfulness limitations of the widely used Logit Lens technique. By combining attributions with semantic alignment scores, we can automatically uncover reliance on components that encode semantically unexpected or spurious concepts. Applied across multiple CLIP variants, our method uncovers hundreds of surprising components linked to polysemous words, compound nouns, visual typography and dataset artifacts. While text embeddings remain prone to semantic ambiguity, they are more robust to spurious correlations compared to linear classifiers trained on image embeddings. A case study on skin lesion detection highlights how such classifiers can amplify hidden shortcuts, underscoring the need for holistic, mechanistic interpretability. We provide code at https://github.com/maxdreyer/attributing-clip.
