Search papers, labs, and topics across Lattice.
This paper introduces BlueMagpie-TTS, a novel text-to-speech (TTS) system specifically designed for Taiwanese Mandarin, addressing significant shortcomings in existing systems that struggle with accent adaptation and tokenization for code-switching scenarios. By leveraging PangolinTokenizer, a byte-level BPE tokenizer optimized for Taiwan-context data, and Barbet, a billion-parameter language model, the authors achieve substantial improvements in character error rate (CER) and word error rate (WER) on a localized test set. The results show a remarkable reduction in CER from 11.45% to 4.81% and WER from 14.83% to 5.36%, alongside strong listener preference in a blind study, highlighting the effectiveness of their approach.
Taiwanese Mandarin TTS systems can achieve a 63.9% reduction in word error rates by using a context-adapted tokenizer and language model.
Off-the-shelf TTS systems are poorly adapted to Taiwanese Mandarin. Their accent defaults to other Mandarin variants, their tokenizers over-segment common Taiwanese text, and their pronunciation degrades at code-switching boundaries where Chinese and English alternate within one utterance. These problems share one root: the text side lacks adaptation to the Taiwanese context. We address the text side from the bottom up. PangolinTokenizer, a byte-level BPE tokenizer trained on Taiwan-context data, reaches the lowest token rate (0.485 tokens/character) with the smallest vocabulary among nine tokenizers. Barbet, a billion-parameter Traditional-Chinese language model trained on PangolinTokenizer, serves as the text-semantic frontend and ranks first among comparable public models on a 14-task evaluation. BlueMagpie-TTS attaches Barbet to the pretrained acoustic stack of VoxCPM2 through a learned bridge, keeping the acoustic stack fixed. On a 1000-sentence Taiwan-localized test set, it lowers CER from 11.45% to 4.81% and WER from 14.83% to 5.36%, relative reductions of 58.0% and 63.9%. In a blind listening study on 500 of these sentences with ten listeners, 65.6% of majority votes prefer BlueMagpie-TTS.
