Search papers, labs, and topics across Lattice.
UniPASE, a universal speech enhancement model, extends the PASE framework to handle diverse distortions across multiple sampling rates. It uses a DeWavLM-Omni module, fine-tuned via knowledge distillation, to convert degraded waveforms into clean phonetic representations, minimizing linguistic hallucination. An adapter then generates enhanced acoustic representations for a neural vocoder to reconstruct high-fidelity waveforms, achieving state-of-the-art performance on several evaluation datasets.
Achieve state-of-the-art universal speech enhancement with UniPASE, a model that minimizes linguistic hallucinations by converting degraded waveforms into clean phonetic representations.
Universal speech enhancement (USE) aims to restore speech signals from diverse distortions across multiple sampling rates. We propose UniPASE, an extension of the low-hallucination PASE framework tailored for USE. At its core is DeWavLM-Omni, a unified representation-level enhancement module fine-tuned from WavLM via knowledge distillation on a large-scale supervised multi-distortion dataset. This module directly converts degraded waveforms into clean and linguistically faithful phonetic representations, ensuring robust enhancement with minimal linguistic hallucination. Based on these enhanced phonetic representations, an Adapter generates enhanced acoustic representations containing rich acoustic details, which a neural Vocoder uses to reconstruct corresponding high-fidelity 16-kHz waveforms. A PostNet then converts the waveforms to 48~kHz before resampling them to their original rates, enabling seamless handling of inputs and outputs at multiple sampling rates. Experimental results on several evaluation datasets, covering sub-tasks and full tasks, demonstrate that UniPASE achieves superior or competitive performance compared with existing state-of-the-art models. The proposed model also serves as the backbone of our submission to the URGENT 2026 Challenge, which achieved 1st place in the objective evaluation. The source code and audio demos are available at https://github.com/xiaobin-rong/unipase/.
