Search papers, labs, and topics across Lattice.
This paper introduces Speech Representation Fr茅chet Distance loss (SR-FD), a novel training-time distributional regularizer for tokenizer-free flow-matching autoregressive text-to-speech (TTS) models. By aligning the mean and covariance of synthesized speech features with reference statistics from high-quality speech, SR-FD significantly enhances the intelligibility of TTS outputs. The method achieved a 36.5% relative reduction in word error rate (WER) on Seed-TTS English, outperforming both the original four-step VoxCPM2 baseline and a ten-step baseline, while maintaining speaker similarity and quality.
SR-FD reduces word error rates by over a third, transforming the landscape of intelligibility in few-step TTS synthesis.
Few-step diffusion and flow-matching text-to-speech (TTS) models are usually trained with local objectives, such as conditional flow matching, reconstruction, and stop prediction. These losses provide stable optimization, but they never ask whether sampled speech follows the distribution of high-quality speech. We propose Speech Representation Fr'echet Distance loss (SR-FD), a training-time distributional regularizer for tokenizer-free flow-matching autoregressive TTS. During fine-tuning, the model synthesizes speech with the same few-step sampler used at deployment, and SR-FD matches the mean and covariance of frozen Whisper and CTC features of this speech to reference statistics computed offline from three complementary content targets. The loss requires no discriminator and no inference-time computation. On Seed-TTS English, four-step SR-FD fine-tuning reduces WER from the original four-step VoxCPM2 baseline's 2.2279% to 1.4147%, a 36.5% relative reduction, and also surpasses the original ten-step baseline at 1.7366%; both gains are significant under an utterance-level paired bootstrap. Speaker similarity and objective quality proxies are preserved at the ten-step level, and an error analysis shows the gain comes from content substitutions across all prompt lengths. SR-FD is thus an intelligibility-improving distributional regularizer for few-step TTS.
