Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces SoulX-Duplug, a plug-and-play module for full-duplex spoken dialogue systems that performs streaming ASR and leverages textual information for user intent identification, acting as a semantic voice activity detector (VAD). To address the lack of fair evaluation datasets, the authors also present SoulX-Duplug-Eval, an extension of existing benchmarks with improved bilingual coverage. Experiments demonstrate that SoulX-Duplug enables low-latency streaming dialogue state control and outperforms existing full-duplex models in turn management and latency.
Achieve human-like full-duplex voice interactions with SoulX-Duplug, a plug-and-play module that slashes latency and improves turn management by acting as a semantic VAD.
Recent advances in spoken dialogue systems have brought increased attention to human-like full-duplex voice interactions. However, our comprehensive review of this field reveals several challenges, including the difficulty in obtaining training data, catastrophic forgetting, and limited scalability. In this work, we propose SoulX-Duplug, a plug-and-play streaming state prediction module for full-duplex spoken dialogue systems. By jointly performing streaming ASR, SoulX-Duplug explicitly leverages textual information to identify user intent, effectively serving as a semantic VAD. To promote fair evaluation, we introduce SoulX-Duplug-Eval, extending widely used benchmarks with improved bilingual coverage. Experimental results show that SoulX-Duplug enables low-latency streaming dialogue state control, and the system built upon it outperforms existing full-duplex models in overall turn management and latency performance. We have open-sourced SoulX-Duplug and SoulX-Duplug-Eval.