Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to speech denoising by leveraging the concept of noise inseparability, allowing for weakly-supervised training using noisy targets. By estimating both the artificial noise and the naturally noisy speech, the authors exploit the residual noise to enhance denoising performance, effectively enabling joint training with conventional artificial mixtures. The proposed method shows significant improvements in domain adaptability and performance on WHAM! and CHiME-3 benchmarks, addressing the limitations of traditional supervised training methods.
Exploiting noise inseparability allows for a breakthrough in weakly-supervised speech denoising, enhancing domain adaptability and performance without relying on clean targets.
Speech denoising is an often necessary step not only for human listening, but also for downstream processing by systems lacking robustness to noisy, real-world acoustic conditions. Unfortunately, denoising is a problem where conventional in-domain supervised training is not trivial, as the training targets cannot be annotated by humans: producing a clean version of a naturally-noisy speech recording is itself the task to solve. Supervised training is typically performed through the artificial addition of noise to clean speech recordings, which can only be sourced from controlled domains, a significant limitation due to the poor out-of-domain generalization of neural networks. An alternative is noisy target training (NyTT), which simply replaces the clean speech with in-domain noisy recordings, with the hope that learning to remove the artificial noise will extend to the natural. Though having shown promising results, NyTT's training objective is not minimized by clean speech estimates. We show that by estimating the artificial noise in addition to the naturally-noisy speech, the undesirable optimum can actually be exploited: the residual noise in the speech estimate can be canceled by the noise estimate via simple subtraction. Crucially, the optimum is fully compatible with conventional artificial mixtures, enabling joint training using both types of data with consistent optimization targets, opening the door to improved domain adaptability. The effectiveness of our approach is demonstrated through WHAM! and CHiME-3-based benchmarks.
CMU ML