Search papers, labs, and topics across Lattice.
Bagpiper-TTS is a universal speech synthesis system that interprets natural language prompts to generate comprehensive captions, which guide the synthesis of speech. This approach overcomes the limitations of traditional TTS systems by accommodating diverse user requests and supporting a wide range of applications, including multi-talker and singing voice synthesis. Experimental results show that Bagpiper-TTS achieves a 1.7% Word Error Rate on the Seed-TTS-Eval benchmark, matching the performance of specialized models in both automated and human evaluations.
Bagpiper-TTS can seamlessly transform natural language requests into high-quality speech across diverse applications, outperforming traditional TTS systems.
Classical TTS systems typically rely on rigid input formats and predefined metadata slots, limiting their ability to fulfill flexible user requirements. This paper introduces Bagpiper-TTS, a universal speech synthesis system that deals with diverse natural language user requests. Given a natural language prompt, Bagpiper-TTS first reasons over the users' intent to derive a rich caption, i.e., a comprehensive textual blueprint encompassing both transcription and nuanced metadata. Subsequently, this caption guides the synthesis of the target speech. Our model inherently supports a broad spectrum of tasks besides classical TTS applications, including multi-talker, intent-to-speech, role-play synthesis, singing voice synthesis, and more. Experimental results demonstrate that Bagpiper-TTS achieves an 1.7% Word Error Rate (WER) on the Seed-TTS-Eval benchmark and match the performance of dedicated models in both LLM-as-a-judge and human subjective evaluations across multiple applications.
CMU ML
NVIDIA