Search papers, labs, and topics across Lattice.
This paper introduces the Half-frame-rate Adaptive Learnable Operator (HALO), a novel module designed to optimize STFT-based speech enhancement by halving the internal frame rate while maintaining the integrity of the STFT process. By implementing adaptive rate reduction and restoration using lightweight dynamic convolutions, HALO significantly decreases computational costs associated with overlapping analysis frames without introducing additional latency. Experimental results on the DNS3 dataset reveal that HALO consistently enhances performance across various lightweight models, effectively addressing redundancy in frame processing.
HALO reduces computational costs by 50% without sacrificing performance, revolutionizing lightweight speech enhancement models.
STFT-based speech enhancement typically adopts overlapping analysis frames. While overlap is essential for stable STFT processing, it makes adjacent frames highly correlated, causing redundant computation in lightweight models. We propose Half-frame-rate Adaptive Learnable Operator (HALO), a causal plug-in module that halves the internal frame rate without altering the STFT procedure. Broadly applicable to many lightweight models, HALO applies adaptive rate reduction before the backbone and restoration afterward, reconstructing the full-rate spectrum on the original STFT grid. Both reduction and restoration are implemented with lightweight dynamic convolutions. By halving the processed frame rate, HALO reduces backbone compute cost with no added algorithmic latency, freeing budget for channel widening. Experiments on the DNS3 dataset show consistent gains across diverse lightweight models under matched complexity, demonstrating the effectiveness of reducing overlap-induced redundancy.
