Search papers, labs, and topics across Lattice.
This paper introduces a novel emotional TTS system that incorporates fine-grained control over non-verbal vocalizations (NVs) by curating and annotating a subset of the EARS corpus with tags encoding NV types, frequencies, and durations. They then build an emotional TTS benchmark to demonstrate the effectiveness of their approach. Results show a significant improvement in expressiveness and emotional recognition accuracy, particularly for high-arousal emotions and sadness, with minor trade-offs in perceived naturalness.
Injecting fine-grained non-verbal cues into TTS can almost perfectly convey sadness (98.3%) and significantly boosts expressiveness overall, despite slight naturalness trade-offs.
While current emotional Text-to-Speech (TTS) models have successfully controlled verbal prosody, they often ignore non-verbal vocalizations (NVs), which are essential for authentic human emotion. Although some non-verbal datasets have recently emerged, they often lack high-quality, fine-grained annotations, which restricts a model's ability to precisely control NV generation. To address this limitation, we propose a novel approach for fine-grained non-verbal expression synthesis. We curate and reprocess female NV utterances from the EARS corpus, develop a new annotation scheme using tags to encode NV types, frequencies, and durations, and build an emotional TTS benchmark to demonstrate its effectiveness. Our evaluation shows that while our NV approach leads to minor trade-offs in perceived naturalness, it significantly improves expressiveness (eMOS 4.20) and emotional recognition accuracy (78.8%). Emotion-specific analysis further reveals that NV cues are highly effective for high-arousal emotions like happy (82.5%) and fear (82.7%), and almost perfectly convey sadness (98.3%).
