Search papers, labs, and topics across Lattice.
This paper introduces a classifier-based framework to evaluate multilingual text-to-speech (TTS) systems by auditing their output against language-specific phonological patterns, using human speech as a benchmark. The study specifically examines Assamese advanced tongue root (ATR) vowel harmony in Meta's MMS TTS, revealing that a significant portion of synthesized tokens misrepresent the intended phonological contrasts. The findings highlight a critical gap between the intended and produced phonology in TTS systems, suggesting that traditional evaluation metrics like Mean Opinion Score (MOS) may overlook important phonetic fidelity issues.
A third of synthesized tokens in TTS systems misrepresent phonological contrasts, revealing a significant gap between intended and produced sounds.
Neural TTS systems can sound natural across languages, but naturalness does not guarantee the preservation of sound contrasts that distinguish words from their grammatical forms. Standard metrics like MOS do not test for this. We propose a classifier-based framework that audits TTS output against language-specific phonological patterns using human speech as a benchmark. Testing Assamese advanced tongue root (ATR) vowel harmony with Meta's MMS TTS, we show that a classifier trained on human speech transfers to synthesized speech with minimal loss. The faithfulness audit reveals that [+ATR] mid vowels are realized as [-ATR] in 1/3 tokens despite an underlying [+ATR] specification, a bias absent in human speech. At the word level, predicted ATR labels classify harmony more accurately than transcription labels, indicating a gap between intended and produced phonology. The framework offers task-specific diagnostics and generalizes to other phonological contrasts with measurable acoustic cues.
