Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces PrefSQA, a novel approach for speech quality assessment that shifts from mean opinion scores (MOS) to preference prediction, addressing the challenges of rater variability and labeling noise. By leveraging uncertainty-aware logits and an impairment attention head, the method enhances the reliability of quality assessments through direct signal comparisons. Experimental results indicate that while improvements on MOS-derived data are modest, significant gains are observed with high-quality preference datasets, underscoring the importance of dataset quality in preference prediction tasks.
Preference prediction can significantly enhance speech quality assessments, revealing the critical impact of dataset quality on model performance.
Mean opinion scores (MOS) are widely used for speech quality assessment, yet scalar labels are sensitive to rater variability and listening test differences. This introduces labeling noise, which limits the reliability of MOS prediction. Preference prediction reduces this variability as listeners compare signals directly, producing cleaner labels. We study MOS-free preference prediction and propose PrefSQA, which incorporates uncertainty-aware logits, an impairment attention head, and a module based on non-matching-reference comparisons. We use and refine five datasets, including MOS-derived and low-noise simulated sets with matching and non-matching content, experiment with human preference sets, and test on unseen data. Experiments show small improvements on MOS-derived data, while other sets reveal clear improvement over the baselines, highlighting the value of high-quality preference data and demonstrating the effectiveness of the proposed method.