Search papers, labs, and topics across Lattice.
This paper introduces DELTA-TTS, a novel framework that transforms a pretrained autoregressive (AR) text-to-speech (TTS) model into a discrete diffusion language model (dLLM) to enhance synthesis speed and robustness. By leveraging a convolution module for local acoustic context and a time-shifted inference schedule, DELTA-TTS achieves a 1.75% word error rate (WER) on the Seed-TTS test set while generating tokens 3.3 times faster than its AR counterpart. The results indicate improved text-speech alignment and reduced hallucinations, showcasing a significant advancement in TTS technology.
DELTA-TTS achieves 3.3x faster token generation while reducing word error rates, transforming the landscape of text-to-speech synthesis.
Autoregressive (AR) text-to-speech (TTS) models generate discrete speech tokens sequentially, which makes inference slow and can degrade robustness by propagating local errors and hallucinations. This limitation stems from their left-to-right AR commitment: each token must be determined before future speech-token context is available. However, such ordering is not an inherent requirement for TTS, as the full input text is available before synthesis. In this paper, we introduce DELTA-TTS, a lightweight LoRA-based adaptation framework that converts a pretrained AR TTS model into a discrete diffusion language model (dLLM) for confidence-ordered speech-token decoding. To better capture the local structure of speech, DELTA-TTS incorporates a convolution module that injects local acoustic context, together with a $1/t$-weighted training objective and a time-shifted inference schedule that defer low-confidence positions to later steps. Trained on only $585$ hours of LibriTTS, DELTA-TTS achieves a $\textbf{1.75}\%$ WER on Seed-TTS test-en, outperforming its AR backbone while generating tokens $\textbf{3.3}\times$ faster. Further analysis shows that DELTA-TTS produces sharper text--speech alignment, increases overall decoding confidence, and mitigates hallucinations observed in AR generation.
