Search papers, labs, and topics across Lattice.
This paper investigates the reliability of symbol detection in Concept Bottleneck Models (CBMs) by examining the effects of swapping independently trained concept detectors and classification heads. The authors find that while high task accuracy can be misleading, their proposed reliability-aware training strategy significantly improves the robustness of concept detection, effectively doubling swap accuracy in scenarios where models typically exhibit leakage. This work highlights the importance of ensuring that explanations provided by CBMs are not only interpretable but also reliable, addressing a critical gap in the deployment of explainable AI systems.
Swapping concept detectors in CBMs reveals that high task accuracy can mask significant reliability issues, but a novel training strategy can nearly eliminate this leakage.
Concept Bottleneck Models (CBMs) are a relevant tool for explainable Artificial Intelligence because they make their predictions through human-interpretable symbols. However, high task accuracy does not guarantee that these symbols are detected faithfully: jointly trained CBMs may encode task-specific shortcuts in the bottleneck, making their explanations unreliable. In this paper, we study concept-detection reliability by swapping independently trained concept detectors and classification heads that share the same symbolic vocabulary. We use the resulting performance degradation, concept-level metrics, and symbol-wise uncertainty estimates to identify concepts that are especially prone to spurious firing. Finally, we propose a reliability-aware training strategy in which a shared concept detector is optimized with multiple classification heads and penalized for relying on globally or instance-wise unreliable symbols. On CUB-200-2011 with full concept supervision, detectors and heads are almost freely interchangeable (swap drop below one accuracy point, relative retention above $99\%$, and no concept detected below chance), whereas on a controlled synthetic task we show that, as the concept-supervision weight is reduced, models keep near-perfect task accuracy while swapped accuracy and agreement with the ground-truth concepts collapse to chance. Our reliability-aware training substantially mitigates this leakage, roughly doubling swap accuracy in the leaky regime.
