Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
MeanVoiceFlow2 addresses the inference bottleneck in one-step zero-shot voice conversion by jointly optimizing a flow-based conversion module and a computationally efficient content encoder. This is achieved through conversion distillation from a prior model and reconstruction of real data, augmented with diffusion-GAN training for improved realism. The resulting model achieves superior perceptual quality and a $9\times$ speedup over its predecessor while maintaining comparable speaker similarity.
Zero-shot voice conversion now achieves $9\times$ faster inference with improved perceptual quality by jointly optimizing flow and content encoding.
Flow-matching approaches to voice conversion (VC) have gained attention owing to their high speech quality and strong speaker similarity. Among them, one-step models such as MeanVoiceFlow are particularly attractive because they enable efficient inference; however, their reliance on a computationally intensive content encoder remains a bottleneck. We therefore propose MeanVoiceFlow2, a framework that jointly optimizes a flow-based conversion module and a computationally efficient content encoder. The model is trained through conversion distillation using MeanVoiceFlow and the reconstruction of real data. We further incorporate diffusion-GAN training with sample mixing and teacher-guided conditioning augmentation to enhance realism and disentanglement. Experiments on zero-shot VC showed that MeanVoiceFlow2 achieved higher perceptual quality and approximately $9\times$ faster inference than MeanVoiceFlow while maintaining comparable speaker similarity. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow2/.