Search papers, labs, and topics across Lattice.
This study investigates the interplay between naturalness and appropriateness in text-to-speech (TTS) systems across five distinct domains, revealing that appropriateness can vary independently of naturalness. By evaluating five state-of-the-art TTS systems, the authors find that while systems excel in reading contexts, they struggle with more expressive domains, indicating that optimizing for one aspect can negatively impact another. The research underscores the inadequacy of universal evaluation metrics for TTS, advocating for context-aware assessments to better capture performance nuances across different applications.
Appropriateness in TTS varies significantly across domains, challenging the assumption that naturalness alone suffices for effective evaluation.
Text-to-speech (TTS) evaluation is an open challenge. While the primary target was "naturalness," recent fidelity gains shifted focus toward "appropriateness" and whether speech is correct for its context. In this work, we examine how perception changes when the expected downstream use varies. We measure the appropriateness and human-likeness of five SOTA TTS systems across five domains: AI assistant, reader, actor, animated character, and spontaneous speaker. Results show appropriateness varies across domains independently of naturalness. While systems shine at reading, expressive domains remain challenging, and optimizing for one can degrade others. Furthermore, naturalness scores tend to penalize stylized speech while rewarding spontaneity. Finally, our study also highlights blind spots in one-size-fits-all evaluation metrics across more expressive domains. We demonstrate that TTS performance is not "solved" but depends on the target domain, requiring context-aware evaluation.
