Search papers, labs, and topics across Lattice.
This paper repurposes the XTTSv2 multilingual voice cloning model for speaker anonymization, achieving a balance between privacy and intelligibility without the need for retraining. By leveraging the model's ability to preserve prosodic structure while conditioning on a pseudo-speaker, the authors introduce an iterative refinement strategy that maximizes both speaker dissimilarity and speech intelligibility. The resulting system demonstrates near-optimal privacy and superior speech quality compared to existing anonymization methods across multiple languages.
XTTSv2 can anonymize voices while maintaining high speech quality and intelligibility, achieving near-optimal privacy without any language-specific training.
Speaker anonymization suppresses speaker-identifying attributes from speech while preserving linguistic content and quality. We propose repurposing XTTSv2, a multilingual voice cloning model trained on 27k hours of speech, for speaker anonymization without retraining. Our key insight is that XTTSv2's voice cloning capabilities preserve prosodic structure independently of speaker identity, enabling voice conversion by conditioning on a pseudo-speaker. We introduce an iterative refinement strategy that balances privacy and utility by maximizing a harmonic mean of speaker dissimilarity and intelligibility. Evaluated on seven European languages across CommonVoice and Multilingual LibriSpeech, our system achieves near-optimal privacy (EER $\approx$ 0.49), competitive intelligibility, and substantially better speech quality than dedicated anonymization baselines, while requiring no language-specific training. We release the code here: https://github.com/rm00cr/coqui-tts.
