Investigating Human-Model Discrepancies in Speech Quality Assessment via Acoustic and Prosodic Perturbations

2026-06-18

AI Summary

This study addresses the limitations of current speech quality assessment models in capturing perceptual differences arising from prosodic errors and variations in speaker characteristics such as fundamental frequency and speaking rate. By systematically introducing acoustic degradations, prosodic distortions, and speaker attribute modifications through controlled perturbations, the authors compare human subjective ratings with predictions from state-of-the-art mean opinion score (MOS) models. Their analysis reveals that existing models are generally insensitive to prosodic errors, exhibit artificial biases toward fundamental frequency, and fail to respond adequately to changes in speaking rate and its variability. These findings challenge the prevailing paradigm of relying solely on scalar MOS for speech quality evaluation, highlighting the unidimensionality of current approaches and providing empirical grounding for the development of more perceptually aligned, multidimensional speech quality assessment frameworks.

Abstract

Mean opinion score (MOS) prediction models are widely used as proxy metrics in text-to-speech (TTS) research, yet their ability to capture quality differences beyond acoustic fidelity remains unclear. We investigate this via controlled perturbations on speech: acoustic degradation, prosodic errors, and manipulation of speaker-specific characteristics such as pitch and speaking rate. We obtained MOS predictions for these speech samples from both human listeners and the model, and analyzed the differences in their perceptual characteristics. Results show that most models track acoustic degradation well, while all are insensitive to prosodic errors despite large subjective score drops. For speaker characteristics, models exhibit a double dissociation: strong mean fundamental frequency (F0) biases absent in human ratings, yet insensitivity to speaking rate and F0 variability that humans notice. These findings highlight limitations of scalar MOS prediction beyond acoustic fidelity.

Problem

• speech quality assessment
• mean opinion score
• prosodic errors
• acoustic fidelity
• human-model discrepancy

Innovation

• MOS prediction
• prosodic perturbation
• acoustic degradation
• speaker characteristics
• human-model discrepancy