Song Aesthetics Evaluation with Multi-Stem Attention and Hierarchical Uncertainty Modeling

2026-01-18

AI Summary

Existing approaches struggle to effectively evaluate the aesthetic quality of music, and single-score Mean Opinion Score (MOS) prediction fails to capture the inherent uncertainty in human perception. To address this limitation, this work proposes the first uncertainty-aware framework specifically designed for musical aesthetic assessment. The method introduces a novel multi-track cross-attention mechanism and a hierarchical granularity-aware interval aggregation strategy, leveraging probabilistic distribution modeling and multi-granularity feature fusion to generate multidimensional aesthetic ratings. Experimental results on both AI-generated and human-composed music datasets demonstrate that the proposed approach significantly outperforms current state-of-the-art models, achieving higher accuracy and improved perceptual consistency in aesthetic evaluation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Music generative artificial intelligence (AI) is rapidly expanding music content, necessitating automated song aesthetics evaluation. However, existing studies largely focus on speech, audio or singing quality, leaving song aesthetics underexplored. Moreover, conventional approaches often predict a precise Mean Opinion Score (MOS) value directly, which struggles to capture the nuances of human perception in song aesthetics evaluation. This paper proposes a song-oriented aesthetics evaluation framework, featuring two novel modules: 1) Multi-Stem Attention Fusion (MSAF) builds bidirectional cross-attention between mixture-vocal and mixture-accompaniment pairs, fusing them to capture complex musical features; 2) Hierarchical Granularity-Aware Interval Aggregation (HiGIA) learns multi-granularity score probability distributions, aggregates them into a score interval, and applies a regression within the interval to produce the final score. We evaluated on two datasets of full-length songs: SongEval dataset (AI-generated) and an internal aesthetics dataset (human-created), and compared with two state-of-the-art (SOTA) models. Results show that the proposed method achieves stronger performance for multi-dimensional song aesthetics evaluation.

Problem

• song aesthetics evaluation
• Mean Opinion Score
• music generative AI
• human perception
• aesthetic assessment

Innovation

• Multi-Stem Attention
• Hierarchical Uncertainty Modeling
• Song Aesthetics Evaluation
• Interval Aggregation
• Cross-Attention Fusion