Search papers, labs, and topics across Lattice.
This paper introduces Language-Anchored Decomposition (LAD), a novel post-hoc framework that generates human-readable concepts for deep neural networks without altering the model. By leveraging a large language model to propose a concept vocabulary and employing a modified non-negative matrix factorization approach, LAD ensures that the explanations are both faithful to the model's behavior and interpretable. The results demonstrate that LAD achieves spatially precise and decision-relevant explanations across various benchmarks, maintaining accuracy while providing stable concept names.
LAD uniquely combines interpretability and fidelity, delivering human-readable explanations that are grounded in the model's own feature geometry without any retraining.
Deep neural networks are widely deployed in high-stakes visual applications where interpretability is critical, yet existing explanations face a trade-off: post-hoc concept methods recover factors that are faithful to a model's behavior but unnamed, while naming and by-design methods attach human-readable concepts only by retraining or altering the classifier. We propose Language-Anchored Decomposition (LAD), a post-hoc framework that delivers concepts which are simultaneously named, faithful, and obtained without modifying the model. For each class, a large language model proposes a concept vocabulary that CLIP-based similarity maps localize across image regions. Inverting standard non-negative matrix factorization, LAD fixes these language-grounded maps as the coefficient matrix and learns only a concept basis that reconstructs the frozen encoder's activations, so naming becomes a structural constraint and the model's own feature geometry determines which concepts are retained. Removing this anchor preserves accuracy but collapses attribution faithfulness. Across natural-image, scene, and medical-imaging benchmarks, LAD produces spatially precise explanations that are decision-relevant under both concept insertion and deletion, while uniquely providing stable, human-interpretable concept names.
