Are we there yet? A brief survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges

2024-06-13

arXiv.org

AI Summary

Music Emotion Recognition (MER) faces fundamental challenges including low-quality datasets, ambiguous emotion annotations, and poor cross-dataset generalization. To address these, this work conducts a systematic literature review and meta-analysis, integrating over 30 public datasets and 50 state-of-the-art models into a dynamic, open-source knowledge base (hosted on GitHub)—the first of its kind. We propose a multimodal contrastive framework to quantitatively evaluate modeling efficacy across audio, MIDI, and physiological signals. Our analysis identifies three pervasive bottlenecks: annotation inconsistency, dataset bias, and limited model transferability. Crucially, we establish standardized evaluation protocols for MER, enabling rigorous cross-dataset comparability. The study delivers a benchmark framework and open resources to support reproducible, generalizable music emotion computing—advancing both methodological rigor and empirical scalability in the field.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Evaluation and Analysis
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Deep learning models for music have advanced drastically in recent years, but how good are machine learning models at capturing emotion, and what challenges are researchers facing? In this paper, we provide a comprehensive overview of the available music-emotion datasets and discuss evaluation standards as well as competitions in the field. We also offer a brief overview of various types of music emotion prediction models that have been built over the years, providing insights into the diverse approaches within the field. Through this examination, we highlight the challenges that persist in accurately capturing emotion in music, including issues related to dataset quality, annotation consistency, and model generalization. Additionally, we explore the impact of different modalities, such as audio, MIDI, and physiological signals, on the effectiveness of emotion prediction models. Recognizing the dynamic nature of this field, we have complemented our findings with an accompanying GitHub repository. This repository contains a comprehensive list of music emotion datasets and recent predictive models.

Problem

• Assessing music emotion prediction model accuracy and challenges
• Evaluating dataset quality and annotation consistency issues
• Improving cross-dataset generalization and model interpretability

Innovation

• Survey of music emotion datasets and models
• Evaluation standards and competitions overview
• Challenges in dataset quality and generalization