Search papers, labs, and topics across Lattice.
This paper challenges the prevailing notion that information leakage in concept-based models (CMs) is inherently detrimental, arguing instead that such leakage can be beneficial in real-world scenarios where concept incompleteness is common. The authors demonstrate that by reframing the training objective of CMs, it is possible to harness benign leakage to enhance model accuracy and intervenability. Their findings suggest that embracing certain types of leakage can lead to more practical and interpretable models in complex environments.
Embracing information leakage in concept-based models can enhance accuracy and interpretability, turning a perceived flaw into a powerful asset.
Concept-based models (CMs), deep neural networks that ground their predictions on representations aligned with human-understandable concepts (e.g., "round", "stripes", etc.), have been shown to learn representations that leak concept-irrelevant information. As the traditional narrative goes, this leakage is undesirable and should be eradicated as it leads to uninterpretable models. In this paper, we posit that this conventional view of leakage in CMs is not only ill-posed, as the evidence of how leakage makes a model less interpretable is often inconclusive, but also bound to lead to impractical CMs under common real-world constraints. Specifically, we argue that in real-world settings where concept incompleteness is the norm, some leakage is often necessary for constructing accurate and intervenable CMs. To this end, we propose that there is such a thing as benign leakage and show that, by optimizing a reframing of the typical CM training objective, CMs can encourage and exploit this form of leakage without sacrificing accuracy or intervenability.
