EmoHeal: An End-to-End System for Personalized Therapeutic Music Retrieval from Fine-grained Emotions

2025-09-19

AI Summary

Existing digital mental health tools predominantly employ static, one-size-fits-all strategies, rendering them ineffective at responding to subtle, individualized emotional fluctuations—such as bedtime anxiety affecting 1.5 billion people globally. Method: This study proposes a dynamic “Match–Guide–Target” therapeutic framework that uniquely integrates music therapy theory (GEMS model and iso-principle) with knowledge graph technology to establish an interpretable, fine-grained mapping from 27 emotion categories to parametric musical features. Emotion recognition is achieved via fine-tuned XLM-RoBERTa, while multimodal audiovisual retrieval is supported by the CLAMP3 model. Contribution/Results: A user study (N=40) demonstrated statistically significant improvements in emotional state (M=4.12, p<0.001) and high emotion recognition accuracy (M=4.05, p<0.001). Critically, recognition accuracy strongly correlated with therapeutic efficacy (r=0.72, p<0.001), empirically validating the effectiveness of fine-grained, perception-driven personalized music interventions.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Graph-based Machine Learning
• Humans and AI: Emotional Intelligence

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Existing digital mental wellness tools often overlook the nuanced emotional states underlying everyday challenges. For example, pre-sleep anxiety affects more than 1.5 billion people worldwide, yet current approaches remain largely static and "one-size-fits-all", failing to adapt to individual needs. In this work, we present EmoHeal, an end-to-end system that delivers personalized, three-stage supportive narratives. EmoHeal detects 27 fine-grained emotions from user text with a fine-tuned XLM-RoBERTa model, mapping them to musical parameters via a knowledge graph grounded in music therapy principles (GEMS, iso-principle). EmoHeal retrieves audiovisual content using the CLAMP3 model to guide users from their current state toward a calmer one ("match-guide-target"). A within-subjects study (N=40) demonstrated significant supportive effects, with participants reporting substantial mood improvement (M=4.12, p<0.001) and high perceived emotion recognition accuracy (M=4.05, p<0.001). A strong correlation between perceived accuracy and therapeutic outcome (r=0.72, p<0.001) validates our fine-grained approach. These findings establish the viability of theory-driven, emotion-aware digital wellness tools and provides a scalable AI blueprint for operationalizing music therapy principles.

Problem

• Detecting fine-grained emotions from user text
• Mapping emotions to musical parameters using therapy principles
• Retrieving personalized therapeutic music for mood improvement

Innovation

• Fine-grained emotion detection with XLM-RoBERTa model
• Music mapping via knowledge graph and therapy principles
• Audiovisual retrieval using CLAMP3 for therapeutic guidance