Search papers, labs, and topics across Lattice.
This paper addresses the challenge of reliable turn-taking in multiparty dialogue systems, which often falter in handling overlapping speech and rapid speaker changes. The authors propose a two-stage audio-only pipeline that first identifies potential turn boundaries and then verifies these boundaries to determine whether to hold or shift the speaker floor. Their approach, tested on the VoxConverse dataset, demonstrates significant improvements in shift detection, particularly when enhanced with diffusion-based data augmentation techniques.
Turn-taking accuracy in multiparty dialogues can be dramatically improved with a novel two-stage audio pipeline that leverages diffusion augmentation.
Reliable turn-taking is essential for spoken dialogue systems. However, most existing methods are designed for two-speaker interaction and struggle with realistic multiparty audio containing overlap and rapid speaker changes. We study multiparty turn-taking on the VoxConverse dataset and propose an audio-only two-stage pipeline that separates when to trigger a turn boundary from whether the floor is actually transferring. A fast trigger scans the audio and proposes candidate end-of-turn times, while a lightweight verifier runs only at those times to decide \textsc{Hold} or \textsc{Shift} and support next-speaker prediction. We report results in the full multiparty setting and a controlled dyadic top-2 projection for comparability. We also investigate diffusion-based, label-preserving background-audio mixing as a data augmentation strategy. Results show improved shift detection over a baseline, with further improvements from diffusion augmentation.
