From Daily Song to Daily Self: Supporting Reflective Songwriting of Deaf and Hard-of-Hearing Individuals through Generative Music AI

2026-03-09

AI Summary

This work addresses the challenge that existing technologies struggle to support deaf and hard-of-hearing individuals in sustaining personal narrative and emotional reflection through music. We propose a generative AI–driven approach to daily musical journaling, uniquely integrating generative music AI with long-term diary-style creation to empower this population to engage in routine, sustained self-exploration and emotion regulation through song. Through a user-centered design process involving design workshops, formative research, and a multi-session diary study, we developed SoulNote, a generative AI system tailored to this purpose. Empirical findings demonstrate significant improvements among participants across three dimensions: self-insight, emotion regulation, and everyday attitudes toward emotions and self-care, thereby validating the efficacy and innovative potential of our approach.

Technology Category

• Humans and AI: AI for Accessibility
• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

The rapid advancement of generative AI (GenAI) is expanding access to songwriting, offering a new medium of self-expression for Deaf and Hard-of-Hearing (DHH) individuals. However, emerging technologies that support DHH individuals in expressing themselves through music have largely been evaluated in single-session settings and often fall short in helping users unfamiliar with songwriting convey personal narratives or sustain engagement over time. This paper explores songwriting as an extended, music-based journaling practice that supports sustained emotional reflection over multiple sessions. We introduce SoulNote, a GenAI system enabling DHH to engage in iterative songwriting. Grounded in user-centered design, including a design workshop, a preliminary study, and a multi-session diary study, our findings show that ongoing songwriting with \textit{SoulNote} facilitated emotional growth across three dimensions: self-insight, emotion regulation, and \revised{everyday attitudes toward emotions and self-care}. Overall, this work demonstrates how GenAI can support marginalized communities by transforming creative expression into a daily practice of self-discovery and reflection.

Problem

• Deaf and Hard-of-Hearing
• generative AI
• songwriting
• self-expression
• emotional reflection

Innovation

• Generative AI
• Songwriting
• Deaf and Hard-of-Hearing
• Emotional Reflection
• User-Centered Design