Search papers, labs, and topics across Lattice.
This paper introduces ICON decomposition, a novel approach for assessing the importance of concepts in deep neural networks by quantifying the variance explained by each concept while controlling for others. This method addresses the limitations of traditional concept-based explainability techniques that may misinterpret correlations as causal relationships. The results demonstrate that ICON decomposition outperforms seven baseline methods in accurately recovering concept importance on synthetic data and effectively isolates genuine model dependencies in real-world applications like skin lesion and brain imaging analysis.
ICON decomposition reveals the true reliance of deep models on concepts, debunking misleading correlations that traditional methods often overlook.
Deep neural networks often exploit spurious associations in their training data, a failure known as shortcut learning. Concept-based explainability methods screen for shortcuts by testing whether concepts such as a patient's sex or scanner settings can be decoded from a network layer. Because each concept is evaluated in isolation, these methods can mistake correlations between concepts as evidence that the model uses them. We introduce ICON decomposition, which instead quantifies how much of a layer's variance each concept explains after accounting for all other concepts and the outcome. On synthetic data with known ground truth, ICON recovers concept importance more accurately than seven alternative baseline methods. On skin-lesion and brain-imaging models, it isolates the concepts on which a model genuinely relies, quantifies the representation unexplained by any of the supplied concepts, and yields sparse explanations that we validate by retraining and out-of-distribution testing.
