Search papers, labs, and topics across Lattice.
This paper investigates how speaker characteristics are encoded within individual dimensions of self-supervised learning (SSL) speech features extracted from WavLM. By applying PCA to utterance-averaged representations, the authors identify principal dimensions that correlate with specific speaker attributes. They demonstrate that the first principal component primarily encodes pitch and gender, while other dimensions capture intensity, noise levels, and formant frequencies, and they validate these findings through voice synthesis experiments where manipulating these dimensions allows for control over speaker characteristics.
Unlocking control over speaker characteristics in speech synthesis is as simple as tweaking individual dimensions of self-supervised speech features.
How do speech models trained through self-supervised learning structure their representations? Previous studies have looked at how information is encoded in feature vectors across different layers. But few studies have considered whether speech characteristics are captured within individual dimensions of SSL features. In this paper we specifically look at speaker information using PCA on utterance-averaged representations. Using WavLM, we find that the principal dimension that explains most variance encodes pitch and associated characteristics like gender. Other individual principal dimensions correlate with intensity, noise levels, the second formant, and higher frequency characteristics. Finally, in synthesis experiments we show that most characteristics can be controlled by changing the corresponding dimensions. This provides a simple method to control characteristics of the output voice in synthesis applications.
