Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates how adversarial robustness in attribution can emerge implicitly through the learning dynamics of standard stochastic gradient descent (SGD), linking parameter-space and input-space curvature. They demonstrate this implicit regularization effect across various architectures, datasets, and attribution methods, showing it comes with negligible computational overhead. However, they also prove and empirically validate that this robustness often fails to transfer to attention-based attribution with softmax normalization due to entropy constraints, but can be restored with kernel-based attention.
Standard training can yield adversarially robust attribution maps for your model, but watch out for softmax attention, which can kill this robustness.
The adversarial robustness of attributions is a fundamental requirement for reliable explainability in deep learning, yet existing approaches typically rely on computationally expensive explicit regularization. In this work, we show that attribution robustness can arise implicitly from the learning dynamics of standard stochastic gradient descent. We theoretically motivate this effect through connections between parameter-space and input-space curvature, and validate it across architectures, datasets, and attribution methods, with negligible computational overhead. In contrast, we prove that such robustness gains often does not transfer to attention-based attribution under softmax normalization, due to inherent entropy constraints, and we validate this limitation experimentally. Finally, we show that replacing softmax attention with kernel-based attention restores the robustness gains in transformer models. Our results highlight learning dynamics as a principled and practical mechanism for robust explainability, and reveal fundamental limitations of attention-based attribution under normalization.