Search papers, labs, and topics across Lattice.
The paper introduces ZeroSyl, a training-free method for extracting syllable boundaries and embeddings directly from a frozen WavLM model using L2 norms of intermediate layer features. This approach addresses the challenge of long sequences produced by discrete tokens from self-supervised speech encoders in pure speech language models. ZeroSyl achieves competitive syllable segmentation performance and outperforms prior syllabic tokenizers across lexical, syntactic, and narrative benchmarks, demonstrating better scaling behavior for syntactic modeling compared to finer-grained units.
Skip the multi-stage training: ZeroSyl offers a surprisingly simple, training-free way to extract syllable tokens directly from a frozen WavLM model, outperforming existing methods in spoken language modeling.
Pure speech language models aim to learn language directly from raw audio without textual resources. A key challenge is that discrete tokens from self-supervised speech encoders result in excessively long sequences, motivating recent work on syllable-like units. However, methods like Sylber and SyllableLM rely on intricate multi-stage training pipelines. We propose ZeroSyl, a simple training-free method to extract syllable boundaries and embeddings directly from a frozen WavLM model. Using L2 norms of features in WavLM's intermediate layers, ZeroSyl achieves competitive syllable segmentation performance. The resulting segments are mean-pooled, discretized using K-means, and used to train a language model. ZeroSyl outperforms prior syllabic tokenizers across lexical, syntactic, and narrative benchmarks. Scaling experiments show that while finer-grained units are beneficial for lexical tasks, our discovered syllabic units exhibit better scaling behavior for syntactic modeling.
