Search papers, labs, and topics across Lattice.
This paper addresses the vulnerability of automatic speaker verification (ASV) systems to adversarial perturbations by introducing the Positive-Incentive Noise Predictor (PnP), which reformulates adversarial purification as a learnable noising problem. By leveraging the forward noising process, PnP learns input-adaptive {\pi}-noise to enhance the robustness of ASV systems while maintaining performance on natural speech. Experimental results demonstrate that PnP achieves a competitive balance between defense effectiveness and inference efficiency, with a real-time factor as low as 0.014 across various attack scenarios.
Adversarial purification can be revolutionized by a novel framework that introduces learnable positive-incentive noise, achieving robust ASV performance with minimal latency.
Modern automatic speaker verification (ASV) systems are vulnerable to adversarial perturbations. Diffusion-based purification has recently shown strong effectiveness against such perturbations, but its reverse denoising process requires iterative sampling and leads to high inference latency. We find that the forward noising process provides most of the robustness gain. Motivated by this observation, we reformulate adversarial purification as a learnable noising problem, and propose the Positive-Incentive Noise Predictor (PnP), the first framework that explicitly introduces positive-incentive noise ({\pi}-noise) into the purification task. PnP learns input-adaptive {\pi}-noise and mixes it with the input to improve the robustness of downstream ASV systems. Experiments on four advanced ASV backbones show that PnP effectively defends against adversarial attacks while preserving performance on natural speech. Compared with representative purification baselines, the proposed framework provides a competitive balance among defense effectiveness, impact on genuine utterances, and inference efficiency under white-box, black-box, and defender-aware adaptive attacks, with a real-time factor as low as 0.014. Moreover, PnP can be cascaded with a diffusion denoiser to further improve the perceptual quality of purified utterances. Code and purified audio examples are available at https://eurecom-asp.github.io/pnp/
