Search papers, labs, and topics across Lattice.
RT-SEMamba introduces a novel causal speech enhancement model leveraging causal time-frequency Mamba blocks, which utilize a fixed-size recurrent state per layer for efficient long-form inference. The model employs a progressive knowledge distillation strategy that successfully compresses an 8-layer teacher model into a 1-layer student, achieving improved performance on the Voicebank-DEMAND dataset while maintaining low latency. Key results show the 8-layer model achieving a PESQ score of 3.32 under a 25 ms latency constraint, while the distilled student model surpasses a naive baseline with a PESQ of 3.18, all while delivering a 2.75x speedup over the teacher model.
A single-layer speech enhancement model outperforms naive architectures and achieves competitive quality with a significant speedup through progressive knowledge distillation.
We present RT-SEMamba, a fully causal speech enhancement (SE) model built upon causal time-frequency Mamba blocks. Unlike Transformer-based architectures that rely on a growing key-value cache, Mamba propagates a fixed-size recurrent state per layer, enabling memory- and bandwidth-efficient long-form inference. We further introduce a progressive knowledge distillation (KD) strategy that compresses an 8-layer teacher into a shallow 1-layer student by jointly distilling complex spectral outputs and intermediate representations. On Voicebank-DEMAND, the 8-layer RT-SEMamba achieves 3.32 PESQ with a 25 ms algorithmic latency constraint, and the distilled 1-layer student improves over a naive 1-layer baseline from 3.06 to 3.18 PESQ while preserving the same steady-state RTF, delivering a 2.75x speedup over the teacher. These results demonstrate that state-space models with progressive KD provide a competitive quality-latency trade-off for real-time SE.
