Search papers, labs, and topics across Lattice.
This paper introduces REDnet, a novel recursive encoder and decoder architecture designed to tackle the challenge of separating multi-speaker audio mixtures with unknown speaker counts and variable microphone configurations. By employing a recursive decoding strategy that identifies and separates speakers one at a time, REDnet enhances the model's ability to adapt to diverse input scenarios. The approach achieves state-of-the-art performance across several public datasets, demonstrating significant advancements in speech separation capabilities.
REDnet can effectively separate audio from an unknown number of speakers using variable microphone setups, achieving unprecedented accuracy in challenging scenarios.
We propose $\textit{recursive encoder and decoder}$ (RED) for building a single deep neural network (DNN) model that can separate multi-speaker mixtures containing unknown numbers of speakers and variable numbers of microphones arranged in an unknown geometry, a task that has not been studied yet. The decoder of RED recursively detects whether there are active speakers left and separates one speaker at a time. It is designed to be trained in an end-to-end fashion to improve separation performance. The encoder of RED recursively encodes each microphone channel of the input mixture, sequentially incorporating spatial cues. Combining both, the DNN can be trained to separate mixtures not only with unknown numbers of speakers but also with variable numbers of microphones, achieving state-of-the-art performance on multiple public datasets.
