Search papers, labs, and topics across Lattice.
This study investigates the discrepancies between human listeners and mean opinion score (MOS) prediction models in assessing speech quality, focusing on acoustic degradation, prosodic errors, and speaker-specific characteristics. The results reveal that while models effectively track acoustic degradation, they fail to account for prosodic errors and exhibit biases in evaluating speaker characteristics, particularly fundamental frequency. This highlights significant limitations in current MOS models, suggesting they may not fully capture the nuances of human speech perception.
MOS models miss critical prosodic nuances, leading to a disconnect between human perception and automated assessments in speech quality.
Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech (TTS) research, yet their ability to capture quality differences beyond acoustic fidelity remains unclear. We investigate this via controlled perturbations on speech: acoustic degradation, prosodic errors, and manipulation of speaker-specific characteristics such as pitch and speaking rate. We obtained MOS predictions for these speech samples from both human listeners and the model, and analyzed the differences in their perceptual characteristics. Results show that most models track acoustic degradation well, while all are insensitive to prosodic errors despite large subjective score drops. For speaker characteristics, models exhibit a double dissociation: strong mean fundamental frequency (F0) biases absent in human ratings, yet insensitivity to speaking rate and F0 variability that humans notice. These findings highlight limitations of scalar MOS prediction beyond acoustic fidelity.
