Search papers, labs, and topics across Lattice.
This study explores the effectiveness of continuous and discrete latent space modeling strategies within vector quantisation (VQ)-based neural audio codecs for speech enhancement. By introducing the cNAC-SE and dNAC-SE frameworks, the authors demonstrate that the continuous representation model significantly outperforms its discrete counterpart across various test conditions, achieving superior performance in DNS-MOS metrics. The findings reveal that VQ regularisation enhances robustness through clean-prior-constrained regularisation, suggesting its potential applicability to other continuous modeling techniques.
Continuous latent space models can dramatically outperform discrete token approaches in speech enhancement, achieving state-of-the-art results in robustness and quality.
This work investigates modelling strategies in continuous and discrete latent spaces in the vector quantisation (VQ)-based neural audio codec (NAC) speech enhancement (SE), along with the role of VQ regularisation. We propose cNAC-SE and dNAC-SE frameworks that predict continuous representations and discrete tokens in latent space, respectively. Theoretical analysis and visualisations in latent space are performed to exhibit their inherent modelling mechanisms. Experimental results show that the fully fine-tuned cNAC-SE model consistently outperforms all dNAC-SE variants across diverse test conditions and achieves leading performance among established generative approaches in DNS-MOS metrics. Comparison with the discriminative counterpart shows that VQ enhances robustness through an intrinsic effect of clean-prior-constrained regularisation, independent of discrete token processing. This highlights the transferable value of VQ regularisation to other continuous modelling methods.
