Search papers, labs, and topics across Lattice.
The paper introduces G-MaP-SE, a guided speech enhancement framework that utilizes a Gaussian Mixture Model (GMM) to create a clean-speech embedding prior, addressing the limitations of existing methods that depend on either clean enrollment audio or noisy embeddings. By refining noisy conditioning embeddings through prior matching, G-MaP-SE significantly enhances performance in challenging acoustic environments. Experimental results on the VoiceBank+DEMAND and DNS Challenge 2020 datasets demonstrate that this approach not only outperforms traditional methods but also approaches the performance of an oracle clean-conditioning model without requiring enrollment audio during inference.
G-MaP-SE achieves superior speech enhancement by leveraging a GMM-based clean-speech prior, outperforming noisy conditioning methods and closing in on oracle performance.
Using speaker embeddings as conditioning can strengthen speech enhancement, but most methods either require clean enrollment audio or rely on embeddings extracted from noisy speech, which are fragile under noise and domain shift. We propose G-MaP-SE, a guided enhancement framework that builds a clean-speech embedding prior with a Gaussian Mixture Model (GMM) and refines a noisy conditioning embedding by matching it to this prior. The matched prior embedding is then injected into a time-frequency enhancement backbone via a lightweight gated fusion module. Experiments on VoiceBank+DEMAND and DNS Challenge 2020 datasets show that the proposed prior matching consistently outperforms noisy conditioning and substantially narrows the gap to an oracle clean-conditioning upper bound, while requiring no enrollment audio at inference time. The code, audio samples, and checkpoint are available.
