Search papers, labs, and topics across Lattice.
This paper introduces a novel Token2Wav architecture utilizing MeanFlow in a compressed latent space to achieve one-step audio waveform generation, significantly improving the efficiency of neural audio codecs used in Text-to-Speech systems. By modeling average velocity rather than instantaneous velocity, the proposed method achieves up to a 17脳 improvement in Real-Time Factor (RTF) while maintaining perceptual quality. Additionally, the authors implement refinement strategies that enhance audio fidelity without increasing inference time, addressing the critical quality-speed trade-off in conventional decoders.
Achieving one-step audio waveform generation with a 17脳 speedup while preserving quality could revolutionize TTS systems.
Neural audio codecs are central to modern LLM-based Text-to-Speech (TTS) and multimodal systems. As low-bitrate semantic codecs gain prominence, the Token-to-Waveform (Token2Wav) decoder becomes a bottleneck determining both perceptual quality and system efficiency. Conventional multi-step flow-matching decoders offer superior quality but suffer from high inference latency due to iterative sampling, creating a severe quality-speed trade-off. In this paper, we propose a novel Token2Wav architecture that overcomes this limitation by applying MeanFlow in a highly compressed latent space. By modeling the average velocity rather than the instantaneous velocity field, MeanFlow enables true one-step generation. Operating in the latent domain mitigates the memory and stability issues of waveform-level flows, yielding up to a 17$\times$ improvement in Real-Time Factor (RTF) compared to multi-step baselines with negligible quality degradation. Furthermore, we introduce refinement strategies that mitigate latent mismatch, including decoder-only fine-tuning with the MeanFlow generator frozen and end-to-end joint fine-tuning, improving fidelity without increasing inference-time cost. Code and demo are publicly available.
