Search papers, labs, and topics across Lattice.
This paper introduces Speaker Anonymization (SA) as a novel perturbation mechanism for zero-shot voice conversion, effectively balancing the trade-off between timbre leakage and utility preservation. By leveraging SA, the authors develop a strictly causal, zero-lookahead network that minimizes reliance on future context, thus reducing latency without sacrificing audio quality. The results demonstrate that this approach significantly enhances the performance of streaming voice conversion systems, allowing for real-time applications without the drawbacks of traditional methods.
Speaker Anonymization enables real-time voice conversion without the latency penalties of future context buffering, revolutionizing streaming applications.
Streaming zero-shot voice conversion struggles to disentangle timbre from linguistic content without degrading utility or inflating latency. Current methods rely on information bottleneck (IB) or speaker perturbation. While IB filters out timbre, it discards prosody, forcing models to explicitly inject features like fundamental frequency. This often requires buffering future frames, creating algorithmic lookahead latency. On the other hand, existing perturbation methods largely overlook the crucial trade-off between timbre leakage and utility preservation. Recognizing this neglected trade-off, we find that the inherent objective of Speaker Anonymization (SA) aligns well with balancing these factors. Thus, we introduce SA as a novel perturbation mechanism to explicitly mitigate timbre leakage while retaining prosodic utility. Crucially, SA's robust representations significantly alleviate the generator's reliance on future context, enabling our strictly causal, zero-lookahead network. Audio samples are available at https://amphionteam.github.io/Zero-VC-demo/.
