Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the impact of perturbation choices on relevance estimation in explainable AI (xAI) methods, specifically in the context of flood detection from Synthetic Aperture Radar (SAR) imagery. By systematically varying patch geometry and perturbation types, the authors reveal significant inconsistencies and contradictions in the generated relevance maps, highlighting the sensitivity of perturbation-based methods to parameter settings. The findings stress the necessity for rigorous evaluation of perturbation choices to ensure reliable interpretations of model predictions and explanations.
Perturbation choices can drastically alter relevance maps, leading to ambiguous explanations that challenge the reliability of xAI methods in critical applications like flood detection.
Perturbation-based xAI methods are widely used to analyze the behavior and predictions of deep learning models. By altering input regions and measuring the resulting changes in class probabilities with respect to the original image, they assign relevance scores and generate heatmaps that reflect each region's contribution to the prediction. Despite their apparent simplicity, however, perturbation-based methods are sensitive to parameter choices. In this work, we focus on two key parameters of the perturbation pipeline, namely the patch geometry, including the size and shape of the perturbed regions, and the perturbation type, defined by the replacement scheme. Grounded in the use case of flood detection from Synthetic Aperture Radar imagery, we conduct a comprehensive investigation of how relevance estimation changes under different perturbation settings. Beyond visual inspection of the resulting relevance maps, we evaluate their consistency across perturbation strategies and their faithfulness to the model's reasoning. We demonstrate how different perturbation choices can steer the resulting relevance maps, yielding ambiguous and even contradictory explanations. Our findings emphasize the importance of methodological settings in perturbation-based xAI. They underscore the need to carefully inspect and evaluate perturbation choices and to treat them as an integral part when interpreting explanations, ensuring a robust understanding of both the explanations and model predictions.