Search papers, labs, and topics across Lattice.
This paper tackles the challenge of predicting Mean Opinion Scores (MOS) for speech with varying sampling rates (16-48 kHz) by addressing the limitation of existing self-supervised learning (SSL) models which are typically pre-trained on 16 kHz data. They propose a spectrogram-augmented SSL method with a parallel-branch architecture to incorporate high-frequency features present in higher sampling rates. The authors demonstrate that pre-training on a large 48 kHz dataset followed by fine-tuning on a smaller multi-rate dataset significantly improves the generalization performance of MOS prediction in multi-rate speech assessment.
Overlooking high-frequency information in multi-rate speech can significantly hinder speech quality assessment, but SA-SSL-MOS shows how to leverage it for improved generalization.
Designing a speech quality assessment (SQA) system for estimating mean-opinion-score (MOS) of multi-rate speech with varying sampling frequency (16-48 kHz) is a challenging task. The challenge arises due to the limited availability of a MOS-labeled training dataset comprising multi-rate speech samples. While self-supervised learning (SSL) models have been widely adopted in SQA to boost performance, a key limitation is that they are pretrained on 16 kHz speech and therefore discard high-frequency information present in higher sampling rates. To address this issue, we propose a spectrogram-augmented SSL method that incorporates high-frequency features (up to 48 kHz sampling rate) through a parallel-branch architecture. We further introduce a two-step training scheme: the model is first pre-trained on a large 48 kHz dataset and then fine-tuned on a smaller multi-rate dataset. Experimental results show that leveraging high-frequency information overlooked by SSL features is crucial for accurate multi-rate SQA, and that the proposed two-step training substantially improves generalization when multi-rate data is limited.
