Search papers, labs, and topics across Lattice.
The paper introduces Sommelier, an open-source data processing pipeline designed to generate high-quality, multi-speaker conversational data for training full-duplex speech language models (SLMs). Sommelier addresses challenges in processing natural dialogues, such as overlapping speech and back-channeling, which often lead to diarization errors and ASR hallucinations in standard pipelines. By providing a scalable and robust solution, the authors aim to alleviate the scarcity of suitable training data and facilitate the development of more natural human-computer interaction systems.
Unlock the potential of full-duplex speech language models with Sommelier, a new open-source pipeline that tackles the messy reality of multi-speaker conversations.
As the paradigm of AI shifts from text-based LLMs to Speech Language Models (SLMs), there is a growing demand for full-duplex systems capable of real-time, natural human-computer interaction. However, the development of such models is constrained by the scarcity of high-quality, multi-speaker conversational data, as existing large-scale resources are predominantly single-speaker or limited in volume. Addressing the complex dynamics of natural dialogue, such as overlapping and back-channeling remains a challenge, with standard processing pipelines suffering from diarization errors and ASR hallucinations. To bridge this gap, we present a robust and scalable open-source data processing pipeline designed for full-duplex model.
