Uncertainty Estimation in the Real World: A Study on Music Emotion Recognition

2025-01-20

AI Summary

This study addresses the challenge of subjective annotation uncertainty in music emotion recognition, arising from inter-individual variability. We propose a novel paradigm that jointly models both the central tendency (e.g., mean emotional response) and its associated uncertainty—moving beyond conventional point-estimate regression. We systematically evaluate probabilistic regression losses (e.g., negative log-likelihood), Monte Carlo Dropout, and model ensembling for uncertainty quantification on real-world music emotion datasets. Results show that while central-tendency prediction achieves strong accuracy, existing methods substantially misestimate subjective inter-rater variability, deviating markedly from empirical uncertainty distributions—demonstrating that uncertainty modeling is inherently more challenging than mean prediction. To our knowledge, this work establishes the first uncertainty-aware benchmark for music emotion recognition, revealing fundamental difficulties in capturing human perceptual subjectivity. It provides critical methodological insights and cautionary guidance for developing interpretable, robust emotion recognition systems.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Any data annotation for subjective tasks shows potential variations between individuals. This is particularly true for annotations of emotional responses to musical stimuli. While older approaches to music emotion recognition systems frequently addressed this uncertainty problem through probabilistic modeling, modern systems based on neural networks tend to ignore the variability and focus only on predicting central tendencies of human subjective responses. In this work, we explore several methods for estimating not only the central tendencies of the subjective responses to a musical stimulus, but also for estimating the uncertainty associated with these responses. In particular, we investigate probabilistic loss functions and inference-time random sampling. Experimental results indicate that while the modeling of the central tendencies is achievable, modeling of the uncertainty in subjective responses proves significantly more challenging with currently available approaches even when empirical estimates of variations in the responses are available.

Problem

• Music Emotion Recognition
• Interpersonal Perception Uncertainty
• Effective Measurement

Innovation

• Uncertainty Assessment
• Probabilistic Computing
• Inter-individual Variability in Music Perception