Search papers, labs, and topics across Lattice.
This paper introduces SIGMA, a novel Saliency-Guided Sparse Mask Attack designed to enhance the effectiveness and interpretability of adversarial attacks on Speech Emotion Recognition (SER) systems. By leveraging post-hoc explainable AI techniques to generate saliency maps, SIGMA identifies critical areas for perturbation while maintaining constraints on both sparsity and magnitude. The results demonstrate that SIGMA achieves competitive attack success rates across multiple datasets, effectively balancing attack efficacy with consistent explanations of model vulnerabilities.
Adversarial attacks on Speech Emotion Recognition can now be both effective and interpretable, thanks to SIGMA's innovative saliency-guided approach.
Speech conveys rich emotional information. As Speech Emotion Recognition (SER) is usually deployed in privacy-sensitive and reliability-critical environments, adversarial attacks on SER have attracted increasing attention. Existing sparse attacks control the number of perturbed elements, yet, they often lack explainability guidance and explicit measures of explanation consistency. A unified treatment of sparsity and magnitude constraints is also uncommon. In addition, transferability across attack families and target models remains limited. Hence, we propose a SalIency-Guided sparse Mask Attack (SIGMA). On self-supervised speech features, we use post-hoc explainable artificial intelligence (XAI) techniques to produce saliency maps and identify the scope of the mask, and then restrict magnitude-bounded updates to this mask. The mask is computed once and can be reused across models and different sparsity attacks to amortise cost. We evaluate on the IEMOCAP and TESS datasets. Under matched budgets and across multiple sparse-attack settings, SIGMA maintains competitive attack success rates, navigating a conscious trade-off between attack efficacy and explanation consistency. SIGMA therefore provides an efficient and interpretable framework for analysing the vulnerability and explanation behaviour of SER models under structured perturbations.
