Search papers, labs, and topics across Lattice.
This paper introduces SSL-GMMVC, a voice conversion method that leverages Gaussian mixture models to perform transformations in a self-supervised speech representation space. By modeling paired source-target features and utilizing a posterior-weighted sum of affine transforms, the approach achieves improved speaker similarity while maintaining intelligibility and naturalness. Notably, even a constrained covariance variant outperforms deep learning baselines as the number of mixture components increases, linking component selection to phonetic structures and revealing interpretable transformations.
Voice conversion just got a boost鈥擲SL-GMMVC achieves superior speaker similarity while remaining interpretable and analytically tractable.
We introduce SSL-GMMVC, an interpretable voice conversion method in self-supervised speech space. The method models paired source-target features with a Gaussian mixture model and performs conversion as a posterior-weighted sum of affine transforms. This yields locally linear transformations that adapt to heterogeneous feature-space structure while remaining analytically tractable. Through objective and subjective evaluations, we show that SSL-GMMVC improves speaker similarity with comparable intelligibility and naturalness, and that even a constrained covariance variant surpasses a deep learning baseline as the number of mixture components increases. Further analyses link component selection to phonetic structure and reveal interpretable scaling and rotation in the learned transforms. These findings highlight SSL-GMMVC as an effective, analyzable framework for voice conversion.
