Search papers, labs, and topics across Lattice.
This paper presents a flow-matching based text-to-speech (TTS) model that simulates the Lombard effect, allowing for continuous and disentangled control of vocal effort and articulation. By incorporating pseudo-labels for vocal effort and articulation, the model enhances clarity-related acoustic features and enables word-level emphasis to improve intelligibility in challenging environments. Experimental results indicate that the model effectively mimics human speech clarity gains in noisy conditions, showcasing its potential for improving TTS systems in real-world applications.
Speech synthesis can now adaptively enhance clarity and vocal effort, mimicking human responses in noisy environments.
Humans tend to speak louder and clearer in challenging environments, such as noisy conditions or when addressing hearingimpaired listeners, which is called Lombard effect. To simulate this behavior in speech synthesis systems, we introduce a flow-matching based text-to-speech (TTS) model trained with vocal effort and articulation pseudo-labels. The proposed model achieves continuous and disentangled control of vocal effort and articulation, while also enabling word-level emphasis for clarifying specific segments of an utterance. Experimental results show that these control mechanisms effectively improve clarityrelated acoustic features. Furthermore, speech-in-noise experiments demonstrate that our model successfully simulates the intelligibility gains of human clear speech in noisy conditions.
CMU ML