Search papers, labs, and topics across Lattice.
This paper introduces a phoneme-level analysis framework for explainable speech deepfake detection, enhancing interpretability of model predictions by linking them to phonetic units. By employing Gradient-weighted Class Activation Mapping alongside speech recognition, the method generates saliency maps that highlight phonetic cues indicative of spoofed speech. Experimental results demonstrate that this approach achieves detection performance on par with existing systems while offering valuable linguistic insights into the nature of deepfakes across different speakers and conditions.
Phoneme-level insights reveal the hidden linguistic cues that differentiate real speech from deepfakes, enhancing trust in detection systems.
As the accuracy of speech deepfake detection improves with the use of self-supervised representations such as wav2vec 2.0 and HuBERT, understanding why the speech is classified as bona fide or deepfake remains an open challenge. In pursuit of more trustworthy and interpretable artificial intelligence, we introduce a phoneme-level analysis framework that connects model predictions to measurable phonetic units. Our post-hoc explainability method is generally applicable to a variety of speech deepfake detection systems based on convolutional neural networks since it leverages Gradient-weighted Class Activation Mapping in conjunction with speech recognition to generate saliency maps aligned with phonemes and pauses. This pipeline reveals statistically significant attack- and speaker-dependent phonetic cues associated with spoofed speech in terms that humans can understand. Experiments using ASVspoof 5 show comparable detection performance to similar architectures while providing linguistic interpretations across speakers and spoofing conditions.
