Search papers, labs, and topics across Lattice.
This paper introduces Faster IndexTTS-2, an accelerated version of the IndexTTS-2 autoregressive text-to-speech model, which utilizes NVIDIA TensorRT and TensorRT-LLM to enhance inference speed for production deployment on GPUs. The model achieves up to 5.0脳 speedup for the autoregressive GPT component and 3.6脳 end-to-end speedup while maintaining minimal degradation in key performance metrics such as word error rate and naturalness. By enabling streaming synthesis and efficient batching, this work significantly improves the practicality of deploying high-quality TTS systems in latency-sensitive applications.
Achieving up to 5.0脳 speedup in autoregressive TTS synthesis without sacrificing quality could revolutionize real-time voice applications.
Autoregressive text-to-speech models achieve strong naturalness but suffer from slow inference due to sequential token generation, limiting their deployment in production applications that require low latency. IndexTTS-2 is a state-of-the-art autoregressive TTS model consisting of a GPT, a flow-matching Diffusion Transformer, and a vocoder. Despite its high synthesis quality, its inference speed barely reaches real-time without streaming or batching support. We present Faster IndexTTS-2, which accelerates all neural network components of IndexTTS-2 for production deployment on GPUs using NVIDIA TensorRT and TensorRT-LLM. Faster IndexTTS-2 also enables streaming synthesis for latency-sensitive interactive applications, and batched inference across all components to maximize GPU utilization. Experiments on the Seed-TTS benchmark for both English and Chinese demonstrate up to 5.0$\times$ speedup on the autoregressive GPT and 3.6$\times$ end-to-end, with minimal degradation in word error rate, speaker similarity, and naturalness. Our methodology provides a practical reference for efficiently accelerating similar autoregressive speech models on GPUs.
