Search papers, labs, and topics across Lattice.
This paper addresses the lack of standardization and transparency in XAI metric evaluation by proposing XAI Evaluation Cards, a structured documentation template akin to model cards. These cards aim to standardize reporting by explicitly declaring target properties, grounding levels, assumptions, validation evidence, gaming risks, and failure cases for XAI evaluation metrics. The authors argue that widespread adoption of these cards will improve the consistency, comparability, and accountability of XAI research.
Stop reinventing the wheel (or worse, comparing apples to oranges) in XAI evaluation: a standardized "XAI Evaluation Card" could finally bring clarity and rigor to a fragmented field.
The evaluation of explainable AI (XAI) methods is affected by a lack of standardization. Metrics are inconsistently defined, incompletely reported, and rarely validated against common baselines. In this paper, we identify transparency of evaluation reporting as a central, under-addressed problem. We propose the XAI Evaluation Card, a documentation template analogous to model cards, designed to accompany any study that introduces an XAI evaluation metric. The card covers explicit declaration of target properties, grounding levels, metric assumptions, validation evidence, gaming risks, and known failure cases. We argue that adopting this template as a community norm would reduce evaluation fragmentation, support meta-analysis, and improve accountability in XAI research.
