Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces StellarTTS, a novel non-autoregressive text-to-speech framework that leverages a sparse temporal embedding strategy to enhance control over phoneme duration, pronunciation, and prosody. By employing a semantic-aware codec and a lightweight masked generative transformer with only 83 million parameters, StellarTTS achieves a remarkable real-time factor of 0.08 while significantly reducing latency and improving robustness compared to existing TTS systems. The results indicate that StellarTTS maintains competitive audio quality and prosodic naturalness, addressing critical trade-offs in TTS technology.
StellarTTS achieves unprecedented low-latency speech synthesis without sacrificing prosodic naturalness, setting a new standard for TTS systems.
The trade-off between robustness, latency, and prosody critically challenges text-to-speech (TTS) systems. Autoregressive models, despite fidelity, are slow and error-prone; non-autoregressive (NAR) alternatives, while fast, often sacrifice prosodic naturalness via rigid alignments. This paper introduces StellarTTS, a novel mobile-optimized NAR TTS framework based on a sparse temporal embedding strategy, enabling granular control of phoneme duration, pronunciation, and prosody. Furthermore, we propose a semantic-aware codec that facilitates efficient single-stage decoding. Conditioned on the sparse temporal embedding, our 83M-parameter lightweight masked generative transformer achieves a real-time factor (RTF) of 0.08. Experiments demonstrate that StellarTTS attains lower latency and stronger robustness compared to state-of-the-art TTS systems, while maintaining competitive performance in audio quality, prosodic naturalness, and speaker similarity.