Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel voice conversion framework that leverages K-Nearest Neighbors (KNN) retrieval on WavLM representations to create synthetic training pairs from non-parallel speech data. By utilizing real target audio for ground-truth outputs and incorporating a speaker loss from a pretrained speaker verification model, the approach effectively maintains target-speaker identity across multilingual contexts without the need for parallel corpora. Experimental results show that this method not only achieves high naturalness and speaker similarity but also outperforms existing voice conversion baselines, even when trained solely on English data.
Achieving high-quality voice conversion across multiple languages without the need for parallel datasets could revolutionize multilingual speech applications.
We present a voice conversion (VC) framework that utilizes K-Nearest Neighbors (KNN) retrieval over WavLM representations to align non-parallel source and target speech, constructing synthetic training pairs for supervised learning. The retrieved segments serve as synthetic inputs, while real target audio provides ground-truth outputs, forming a synthetic-to-real training paradigm that naturally supports multilingual data without requiring parallel corpora or explicit alignment. To ensure consistent target-speaker identity, we incorporate a speaker loss derived from a pretrained speaker verification model. Experiments across multiple languages demonstrate that the proposed approach achieves high naturalness and strong speaker similarity, outperforming competitive VC baselines, despite being trained exclusively on English data. Samples can be accessed at: https://palindromic-vc.github.io.