Search papers, labs, and topics across Lattice.
This paper introduces a novel inversion-based generative attack framework that aligns the latent space of a speech synthesis model with the discriminative feature space of speaker recognition systems (SRSs) to improve score-based impersonation attacks. By geometrically synchronizing latent representations with speaker embeddings, the method ensures that latent updates directly translate into score improvements, addressing limitations of prior latent-space optimization approaches. Experiments demonstrate a significant improvement in query efficiency, achieving competitive attack success rates with 10x fewer queries, and enabling a subspace-projection-based attack with up to 91.65% success using only 50 queries.
Forget brute-force audio attacks: this new method aligns generative model latent spaces with speaker embeddings, letting you impersonate voices with 10x fewer queries to speaker recognition systems.
Advances in deep learning have enabled the widespread deployment of speaker recognition systems (SRSs), yet they remain vulnerable to score-based impersonation attacks. Existing attacks that operate directly on raw waveforms require a large number of queries due to the difficulty of optimizing in high-dimensional audio spaces. Latent-space optimization within generative models offers improved efficiency, but these latent spaces are shaped by data distribution matching and do not inherently capture speaker-discriminative geometry. As a result, optimization trajectories often fail to align with the adversarial direction needed to maximize victim scores. To address this limitation, we propose an inversion-based generative attack framework that explicitly aligns the latent space of the synthesis model with the discriminative feature space of SRSs. We first analyze the requirements of an inverse model for score-based attacks and introduce a feature-aligned inversion strategy that geometrically synchronizes latent representations with speaker embeddings. This alignment ensures that latent updates directly translate into score improvements. Moreover, it enables new attack paradigms, including subspace-projection-based attacks, which were previously infeasible due to the absence of a faithful feature-to-audio mapping. Experiments show that our method significantly improves query efficiency, achieving competitive attack success rates with on average 10x fewer queries than prior approaches. In particular, the enabled subspace-projection-based attack attains up to 91.65% success using only 50 queries. These findings establish feature-aligned inversion as a key tool for evaluating the robustness of modern SRSs against score-based impersonation threats.
