Search papers, labs, and topics across Lattice.
This paper benchmarks the effectiveness of 13 XAI attribution methods across various architectures, including CNNs and Vision Transformers (ViTs), using a controlled framework that evaluates five dimensions of attribution quality. The findings reveal that attribution performance is highly dependent on the architecture, with rankings from CNNs not transferring reliably to transformer-based models. Notably, while CAM-based methods excel in conventional localization metrics on CNNs and many ViTs, they falter on linear-attention architectures, underscoring the necessity for architecture-aware evaluation in XAI.
Attribution performance varies dramatically across architectures, challenging the assumption that CNN-based evaluations apply to Vision Transformers.
Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural networks (CNNs), with limited investigation into whether these conclusions generalize to the diverse Vision Transformer (ViT) architectures that now dominate computer vision. This paper presents a controlled benchmark that evaluates attribution quality across five dimensions: faithfulness, localization, robustness, complexity, and computational cost. A standardized framework assesses 13 attribution methods from four algorithmic families on eight representative backbones spanning CNNs, isotropic ViTs, hierarchical transformers, hybrid architectures, and linear-attention transformers. The results show that attribution performance is strongly architecture-dependent and that rankings established on CNNs do not reliably transfer to transformer-based models. CAM-based methods achieve the highest scores under the conventional bounding-box localization metric on CNNs and most ViTs but perform poorly on linear-attention architectures. Pixel-level dense-mask evaluation further reveals that these gains largely reflect metric saturation rather than accurate localization. CAM-based methods also exhibit limited robustness on global-attention transformers, whereas attention rollout provides consistently stable explanations with poor localization. Furthermore, faithfulness correlation offers limited discrimination between attribution methods, highlighting the limitations of single-metric evaluation. These findings challenge prevailing conclusions on attribution performance and demonstrate the need for architecture-aware, multi-dimensional evaluation. The open-source code for the evaluation framework and benchmark results is available at https://github.com/Nishan-Charlie/VIT_XAI_Bench.
