Search papers, labs, and topics across Lattice.
This paper investigates whether human articulatory constraints impact end-to-end text-to-speech (TTS) models by training Tacotron2 and VITS-TTS architectures on forward text/speech, reversed text/speech, and reversed text/forward speech data. The study aims to determine if TTS models implicitly learn anatomical limitations during speech production. The key finding is that TTS systems are purely data-driven and surprisingly, reversed text/speech training (r-e2e-TTS) results in improved speech fidelity, intelligibility, and naturalness compared to conventional training.
Surprisingly, end-to-end TTS models trained on reversed speech and text outperform those trained on conventional forward data in terms of fidelity, intelligibility, and naturalness.
An end-to-end (e2e) text-to-speech (TTS) system is a deep architecture that learns to associate a text string with acoustic speech patterns from a curated dataset. It is expected that all aspects associated with speech production, such as phone duration, speaker characteristics, and intonation among other things are captured in the trained TTS model to enable the synthesized speech to be natural and intelligible. Human speech is complex, involving smooth transitions between articulatory configurations (ACs). Due to anatomical constraints, some ACs are challenging to mimic or transition between. In this paper, we experimentally study if the constraints imposed by human anatomy have an implication on training an e2e-TTS systems. We experiment with two e2e-TTS architectures, namely, Tacotron-2 an autoregressive model and VITS-TTS a non-autoregressive model. In this study, we build TTS systems using (a) forward text, forward speech (conventional, e2e-TTS), (b) reverse text, reverse speech (r-e2e-TTS), and (c) reverse text, forward speech (rtfs-e2e-TTS). Experiments demonstrate that e2e-TTS systems are purely data-driven. Interestingly, the generated speech by r-e2e-TTS systems exhibits better fidelity, better perceptual intelligibility, and better naturalness
