Search papers, labs, and topics across Lattice.
This study explores the impact of segmentation width and cluster size on the performance of Generative Spoken Language Models (GSLM) for speech synthesis and continuation using discrete speech representations. By training K-means models with varying cluster sizes and fixed segment widths, the authors reveal that intelligible and natural speech can be achieved at lower bitrates than previously established baselines. Additionally, the findings indicate that speech continuation quality remains consistent across lower bitrate settings, challenging the necessity of conventional GSLM configurations for effective speech generation.
Lowering bitrate settings in speech synthesis can still yield high-quality, intelligible output, challenging traditional assumptions about GSLM configurations.
Generative Spoken Language Modeling (GSLM) enables text-free speech modeling by training language models (LMs) using discrete speech representations instead of textual transcription. In this paper, we investigate the performance of GSLM on speech synthesis and continuation using discrete speech representations with varying bitrates. We segment speech representations with fixed widths and train K-means models in multiple cluster sizes, resulting in various bitrate settings. We demonstrate that intelligible and natural speech can be synthesized at lower bitrate settings than the baseline. Furthermore, speech continuation quality remains stable at lower bitrates across multiple metrics, suggesting that the conventional GSLM setting may be redundant for effective speech generation. Although LLM-based metrics show higher correlation with human subjective score than conventional metrics, it remains low, highlighting the need for more stable automatic evaluation methods.
