Search papers, labs, and topics across Lattice.
This study investigates the ability of pretrained self-supervised speech recognition models, specifically Wav2Vec2 and HuBERT, to accurately recognize click consonants from low-resource Khoisan languages. By fine-tuning these models on data from G|ui and West !Xoon, the researchers found that the models demonstrated superior recognition accuracy for click consonants compared to more commonly encountered speech sounds. This finding highlights the potential of self-supervised learning to generalize across diverse phonetic inventories, including typologically rare sounds.
Fine-tuned self-supervised speech models can recognize click consonants with greater accuracy than more common speech sounds, challenging assumptions about language representation in AI.
Modern pretrained self-supervised automatic speech recognition models are trained on large-scale audio data to encode speech into contextualized representations. However, their training data are heavily skewed toward high-resource languages with little data from low-resource languages, raising concerns about the potential underrepresentation of typologically uncommon speech sounds such as click consonants primarily found in Khoisan languages. This leads to our central research question: Can these models recognize click consonants as accurately as other speech sounds? To address this question, we fine-tune and compare pretrained self-supervised speech models (Wav2Vec2 and HuBERT) on data from two click-rich Khoisan languages (G|ui and West !Xoon). Our results reveal that the fine-tuned models consistently recognize clicks more accurately than non-clicks, suggesting that self-supervision enables generalization across human speech sounds including rare phonemes.
