Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ZipL-Dialog, a novel approach to zero-shot dialog text-to-speech (TTS) that addresses memory bottlenecks in long-form spoken dialog synthesis by utilizing a 4x time-compressed latent space for flow-matching. By implementing a deterministic mel autoencoder with auxiliary supervision and optimizing a hierarchical downsampling schedule, the method significantly reduces GPU memory usage and enhances inference speed. Experimental results demonstrate an 11.22x reduction in peak GPU memory and a 2.23x increase in inference speed while preserving the naturalness of the generated speech.
Achieving an 11.22x reduction in GPU memory usage for long-form dialog synthesis without sacrificing audio quality could revolutionize TTS applications.
Zero-shot dialog TTS benefits from flow-matching, but minute-scale generation on dense mel-spectrograms causes severe memory bottlenecks, often forcing unnatural chunked synthesis. We propose ZipL-Dialog, which shifts conditional flow-matching into a 4x time-compressed (25 Hz) latent space. To preserve acoustic fidelity under compression, we employ a deterministic mel autoencoder with auxiliary mel-domain supervision and optimize the ZipFormer's hierarchical downsampling schedule. Experiments show that ZipL-Dialog reduces maximum peak GPU memory by 11.22x and accelerates inference by 2.23x over the baseline, substantially lowering the memory footprint of single-pass multi-minute dialog synthesis while maintaining perceptual naturalness.