Dialogue in Resonance: An Interactive Music Piece for Piano and Real-Time Automatic Transcription System

2025-05-22

AI Summary

This study addresses the challenge of simultaneously satisfying compositional structural constraints and improvisational responsiveness in real-time human–computer music interaction. To this end, we propose a balanced human–computer dialogue framework centered on real-time score transcription, which integrates predefined compositional intent with dynamic interactive responses. Methodologically, we develop a low-latency, deep time–frequency modeling–based polyphonic automatic transcription system (end-to-end latency < 80 ms), augmented with MIDI closed-loop control, a score-driven response engine, and an interactive rehearsal protocol. Our approach achieves, for the first time, semantically coherent and artistically expressive responses to complex polyphonic input. Experimental validation confirms the feasibility and expressive power of the composition–interaction co-design paradigm, overcoming key limitations of conventional improvisation-centric interaction models. The work establishes a novel methodology and technical pathway for intelligent, collaborative music-making.

Technology Category

• Humans and AI: Human-Computer Interaction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Intelligent Robots: Human-Robot Interaction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper presents, an interactive music piece for a human pianist and a computer-controlled piano that integrates real-time automatic music transcription into a score-driven framework. Unlike previous approaches that primarily focus on improvisation-based interactions, our work establishes a balanced framework that combines composed structure with dynamic interaction. Through real-time automatic transcription as its core mechanism, the computer interprets and responds to the human performer's input in real time, creating a musical dialogue that balances compositional intent with live interaction while incorporating elements of unpredictability. In this paper, we present the development process from composition to premiere performance, including technical implementation, rehearsal process, and performance considerations.

Problem

• Integrates real-time transcription for piano-computer interaction
• Balances composed structure with dynamic live performance
• Enables musical dialogue through real-time interpretation and response

Innovation

• Real-time automatic music transcription system
• Balanced composed and dynamic interaction framework
• Computer interprets and responds to pianist