Search papers, labs, and topics across Lattice.
This paper introduces CSAVocoder, a causal GAN-based spatial audio vocoder that enhances the conversion of mel-spectrograms into high-fidelity spatial audio waveforms while maintaining real-time performance. By incorporating a Spatial Adaptor that integrates multi-channel mel-spectrograms with dynamic source-listener pose information and utilizing a spatial consistency discriminator, the model effectively preserves inter-channel cues that are often neglected in traditional vocoders. Experimental results demonstrate that CSAVocoder achieves superior spatial fidelity and audio quality compared to existing methods, making it suitable for real-time applications.
CSAVocoder achieves real-time spatial audio generation with enhanced fidelity by effectively integrating dynamic spatial cues, outperforming traditional vocoders.
Spatial audio vocoders are able to convert mel-spectrograms produced by generative models into spatial audio waveforms. Most neural vocoders are designed for monaural audio, and direct extensions to spatial audio can degrade spatial quality by ignoring inter-channel cues. We present CSAVocoder, a causal GAN-based spatial audio vocoder that jointly optimizes waveform fidelity and spatial rendering. Our framework introduces a Spatial Adaptor that fuses multi-channel mel-spectrograms with dynamic source-listener pose information, together with a spatial consistency discriminator that supervises inter-channel cues. To meet real-time requirements, we design a strictly causal, stateful generator that supports efficient streaming inference with constant memory overhead. Experiments on large-scale spatial audio datasets show that CSAVocoder improves spatial fidelity at competitive audio quality and real-time performance.
