Calliope: An Online Generative Music System for Symbolic MultiTrack Composition

2025-04-18

International Conference on Innovative Computing and Cloud Computing

AI Summary

To address the low efficiency of symbolic multi-track music composition and weak human-AI collaboration, this paper introduces the first integrated online co-creation system. Methodologically, it combines an enhanced Multi-Track Music Machine (MMM) model with Web Audio API, MIDI.js, and interactive frontend technologies, enabling MIDI upload, visual editing, bar-wise completion, full-piece generation, streaming DAW integration, and real-time low-latency playback. Key contributions include: (1) the first end-to-end human-AI co-creation closed-loop framework; (2) stable, high-fidelity multi-track MIDI generation entirely within the browser; and (3) support for batch generation, real-time auditioning, and seamless import into mainstream digital audio workstations (DAWs). Experimental evaluation demonstrates significant improvements in compositional efficiency and creative flexibility, establishing a scalable, web-native paradigm for AI-assisted music composition.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

With the rise of artificial intelligence in recent years, there has been a rapid increase in its application towards creative domains, including music. There exist many systems built that apply machine learning approaches to the problem of computer-assisted music composition (CAC). Calliope is a web application that assists users in performing a variety of multi-track composition tasks in the symbolic domain. The user can upload (Musical Instrument Digital Interface) MIDI files, visualize and edit MIDI tracks, and generate partial (via bar in-filling) or complete multi-track content using the Multi-Track Music Machine (MMM). Generation of new MIDI excerpts can be done in batch and can be combined with active playback listening for an enhanced assisted-composition workflow. The user can export generated MIDI materials or directly stream MIDI playback from the system to their favorite Digital Audio Workstation (DAW). We present a demonstration of the system, its features, generative parameters and describe the co-creative workflows that it affords.

Problem

• Assisting users in multi-track symbolic music composition
• Generating partial or complete multi-track MIDI content
• Enhancing workflow via batch generation and DAW integration

Innovation

• Web-based multi-track symbolic music composition
• MIDI editing and batch generation via MMM
• Real-time playback and DAW integration