Search papers, labs, and topics across Lattice.
This study addresses the quality gap in text-to-speech (TTS) systems for low-resource languages by fine-tuning the VoxCPM2 model using Low-Rank Adaptation (LoRA) for Khmer and Korean. By creating a shared, language-tagged corpus of 26 hours and training a LoRA adapter, the authors achieved a significant improvement in native-speaker evaluations for Khmer, with the Mean Opinion Score (MOS) increasing from 3.85 to 4.23. Interestingly, while the best performance for Khmer was observed at rank 64, the model showed no improvement for Korean, indicating that adaptation is most beneficial where the base model's performance is lacking.
Fine-tuning a TTS model with LoRA can boost Khmer speech synthesis quality significantly, while revealing that adaptation may not benefit already well-supported languages like Korean.
Large pretrained text-to-speech (TTS) models sound almost human for well-resourced languages, but much worse for languages that are rare in their training data. We study this quality gap for Khmer and Korean using VoxCPM2, a 2.4B-parameter, tokenizer-free TTS model that joins a MiniCPM-4 language-model backbone with a flow-matching diffusion decoder. We build one shared, language-tagged corpus of about 26 hours and adapt VoxCPM2 with a single Low-Rank Adaptation (LoRA) adapter, trained on both languages at once and added to both the language model and the decoder. The adapter is zero-initialized, so training starts exactly at the original (zero-shot) model. In native-speaker listening tests, the Khmer Mean Opinion Score (MOS) rises from 3.85 to 4.23 with the best adapter (rank 64), a highly significant gain (paired Wilcoxon test, p<0.001), while training only 0.19 to 3.03 percent of the parameters. The automatic loss and the human ratings, however, disagree on the best rank: validation loss is lowest at rank 128, yet MOS peaks at rank 64. The same adapter brings no gain for Korean, a language the base model already handles well, and at a high rank it even degrades quality. Adaptation therefore helps mainly where the base model is genuinely weak.
