Search papers, labs, and topics across Lattice.
The paper introduces CTC-TTS, a dual-streaming text-to-speech (TTS) system that leverages Connectionist Temporal Classification (CTC) for text-speech alignment, replacing traditional GMM-HMM methods. It addresses the limitations of existing LLM-based TTS systems in achieving both low latency and high quality in dual-streaming synthesis by improving text-speech alignment and training sequence design. The proposed bi-word interleaving strategy and CTC aligner enable CTC-TTS to outperform fixed-ratio interleaving and MFA-based baselines in streaming synthesis and zero-shot tasks, with two variants (CTC-TTS-L and CTC-TTS-F) trading off quality and latency.
Ditch the clunky GMM-HMM pipelines: CTC-TTS uses a CTC aligner and bi-word interleaving to achieve better streaming TTS with LLMs.
Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. High-quality dual-streaming TTS depends on accurate text--speech alignment and well-designed training sequences that balance synthesis quality and latency. Prior work often relies on GMM-HMM based forced-alignment toolkits (e.g., MFA), which are pipeline-heavy and less flexible than neural aligners; fixed-ratio interleaving of text and speech tokens struggles to capture text--speech alignment regularities. We propose CTC-TTS, which replaces MFA with a CTC based aligner and introduces a bi-word based interleaving strategy. Two variants are designed: CTC-TTS-L (token concatenation along the sequence length) for higher quality and CTC-TTS-F (embedding stacking along the feature dimension) for lower latency. Experiments show that CTC-TTS outperforms fixed-ratio interleaving and MFA-based baselines on streaming synthesis and zero-shot tasks. Speech samples are available at https://ctctts.github.io/.
