Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors systematically evaluate the faithfulness of seven attention- and gradient-based saliency methods on a Vision Transformer trained on the ODELIA breast MRI classification dataset. They investigate a critical reliability gap in clinical interpretability: visually persuasive saliency maps often mask model reasoning unless validated by rigorous perturbation benchmarks. Their results reveal that explainability rankings are deeply unstable across perturbation strategies (intensity alteration vs. attention masking) and frequently confound class-specific with class-agnostic signals, though Grad-CAM and Gradient Attention Rollout consistently rank among the most faithful class-specific methods.
Benchmarking saliency map faithfulness is so sensitive to perturbation design and class-conditioning that minor evaluation choices can completely flip which explainability method appears most trustworthy in medical ViTs.
Saliency maps are widely used to explain deep learning predictions in medical imaging, yet visually plausible explanations do not necessarily reflect a model's true decision process and may therefore mislead clinicians. We investigate this problem using a Vision Transformer-based breast MRI classifier trained on the ODELIA Breast MRI Challenge dataset and evaluate multiple saliency methods, including Last-layer Attention, Attention Rollout, Grad-SAM, Gradient Attention Rollout, GMAR, Grad-CAM, and HiResCAM. Our study highlights two often-overlooked challenges in perturbation-based faithfulness evaluation. First, method rankings depend strongly on the perturbation strategy, varying across intensity-based perturbations and transformer-based attention masking. Second, benchmarking saliency methods requires distinguishing between class-specific and class-agnostic explanations. To enable fair comparisons, we introduce non-class-specific variants of gradient-based methods and evaluate both settings separately. Across protocols, Grad-CAM and Gradient Attention Rollout consistently emerged as the strongest class-specific methods, although their relative ranking depended on the evaluation design. These findings expose important limitations of current saliency-based explainability approaches and highlight the need for more robust and standardized evaluation frameworks for trustworthy clinical AI systems.