Search papers, labs, and topics across Lattice.
This paper introduces DriftSE, a generative speech enhancement framework that formulates denoising as an equilibrium problem solved via a learned "Drifting Field." Unlike iterative diffusion models, DriftSE achieves one-step inference by directly mapping noisy speech to clean speech distributions, enabling training on unpaired data. Experiments on VoiceBank-DEMAND show DriftSE outperforms multi-step diffusion baselines in fidelity while offering faster inference.
Ditch the slow sampling: DriftSE achieves state-of-the-art speech enhancement in a single step, outperforming diffusion models with a novel equilibrium-based approach.
We propose Speech Enhancement based on Drifting Models (DriftSE), a novel generative framework that formulates denoising as an equilibrium problem. Rather than relying on iterative sampling, DriftSE natively achieves one-step inference by evolving the pushforward distribution of a mapping function to directly match the clean speech distribution. This evolution is driven by a Drifting Field, a learned correction vector that guides samples toward the high-density regions of the clean distribution, which naturally facilitates training on unpaired data by matching distributions rather than paired samples. We investigate the framework under two formulations: a direct mapping from the noisy observation, and a stochastic conditional generative model from a Gaussian prior. Experiments on the VoiceBank-DEMAND benchmark demonstrate that DriftSE achieves high-fidelity enhancement in a single step, outperforming multi-step diffusion baselines and establishing a new paradigm for speech enhancement.
