Search papers, labs, and topics across Lattice.
This paper identifies critical shortcomings in current algorithmic explanations, highlighting that they often mislead users due to a lack of intuitive understanding and uninformative outputs. To address these issues, the authors propose "Explanation Cards," which provide essential context on robustness, validity, and interpretation guidelines, effectively bridging the gap between theoretical explanations and practical usability. By demonstrating their application with counterfactual explanations and SHAP, the authors show that these cards enhance user comprehension and align with regulatory requirements like the EU AI Act.
Explanation Cards transform opaque algorithmic explanations into actionable insights, ensuring users understand their limitations and applications.
Algorithmic explanations are intended to help stakeholders understand opaque algorithmic decisions, but in practice, they often fall short. First, the meaning of algorithmic explanations is often not what one might intuitively expect, so expert knowledge is required to interpret them correctly. Second, recent work has shown that popular explanation algorithms are uninformative about the behavior of complex decision functions. Together, these issues create a gap between what explanations appear to convey and what they actually provide. In this work, we propose Explanation Cards for Explanation Algorithms, which augment standard explanations with complementary information about robustness and validity, as well as clear instructions for interpretation. The complementary information can render otherwise uninformative explanations practically useful, while also helping to detect cases where they are not. Importantly, the interpretation instructions in explanation cards shift responsibility from users to providers: Rather than expecting users to recognize what can and cannot be concluded from an explanation, providers must make this explicit upfront. Using counterfactual explanations and SHAP as examples, we demonstrate how providers can construct explanation cards and that these cards provide users with the guidance needed for sound interpretation. We further argue that explanation cards offer a practical means of operationalising the explainability provisions of the EU AI Act. Overall, explanation cards are a significant step toward making explanation algorithms fit for real-world use cases.
