Attacking UTMOS: Probing the Robustness of a Speech Quality Assessment Model

2026-06-30

AI Summary

This study systematically investigates the robustness limitations of the UTMOS speech quality assessment model under adversarial perturbations. To address this issue, we introduce a novel bidirectional adversarial attack framework: one direction generates score-preserving perturbations that degrade perceived quality while keeping the predicted MOS unchanged, and the other produces quality-preserving perturbations that suppress the predicted score without altering perceptual quality. By performing gradient-based optimization in three domains—the raw waveform, HiFi-GAN-reconstructed mel-spectrograms, and EnCodec latent space—we find that attacks in the EnCodec latent space yield the most effective quality-preserving results. Furthermore, we successfully synthesize highly deceptive score-preserving adversarial examples, thereby uncovering critical failure modes and the underlying sources of vulnerability in UTMOS.

Abstract

UTMOS has become one of the most commonly used deep neural network-based speech quality assessment (SQA) metrics in speech processing research. In this paper, we attack UTMOS to probe its robustness. Starting from high-quality speech samples, we optimize the input in two directions: a score-preserving attack, which degrades perceived quality while maintaining the predicted score, and a quality-preserving attack, which lowers the predicted score while maintaining perceived quality. We consider three input spaces: raw waveform, mel spectrogram with a HiFi-GAN vocoder, and the latent space of EnCodec, a neural audio codec. Experimental results show that score-preserving attacks are effective against UTMOS. Although perfect quality-preserving attacks are more difficult, optimization in the EnCodec latent space provides the best chance of success. These results reveal failure modes of UTMOS and highlight the importance of robustness analysis for DNN-based SQA metrics.

Problem

• UTMOS
• speech quality assessment
• robustness
• adversarial attack
• deep neural network

Innovation

• adversarial attack
• speech quality assessment
• UTMOS
• neural audio codec
• robustness analysis