Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a framework for assessing explanation faithfulness in machine learning models, addressing the Rashomon effect where multiple models yield divergent explanations despite similar performance. By employing metamorphic testing, the authors establish five metamorphic relations that evaluate the consistency between model behavior and feature attributions without needing ground-truth labels. The approach is validated on two tabular regression datasets using SHAP and LIME, demonstrating its effectiveness in selecting models that provide reliable explanations.
Trustworthy explanations in machine learning can be assessed without ground-truth labels, revealing the hidden inconsistencies in model interpretations.
Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations. This is called the Rashomon effect of (explainable) machine learning, and it raises the question of which explanations, if any, are trustworthy. We propose a framework based on metamorphic testing that assesses explanation faithfulness without requiring ground-truth labels by exploring attributed feature importance from post-hoc explanation methods. Five metamorphic relations formalize expected consistency properties between model behavior and feature attributions. We apply this general framework to two tabular regression datasets and two post-hoc explainers (SHAP and LIME) to demonstrate the approach. The framework offers a practical, model-agnostic tool for selecting accurate models with reliable and trustworthy explanations.