Hookpad Aria: A Copilot for Songwriters

2025-02-12

AI Summary

This work addresses the nonlinear, interactive composition requirements in popular music creation by proposing a multimodal collaborative generation framework tailored for lead sheets. Methodologically, it introduces a Transformer-based symbolic music generation model that integrates music-theoretic constrained decoding, context-aware masked modeling, and interactive reinforcement learning fine-tuning—enabling bidirectional melody-harmony generation, phrase continuation, and gap-filling, along with fine-grained editing intent understanding. Its key contribution is the novel “user-feedback-driven collaborative co-creation flywheel” mechanism, which closes the loop between real-time user interactions and model iteration. Deployed within the Hookpad editor since March 2024, the system has served 3,000 composers, delivering 318,000 generation suggestions with a 23.3% adoption rate—demonstrating significant improvements in creative efficiency and output quality.

Technology Category

• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

We present Hookpad Aria, a generative AI system designed to assist musicians in writing Western pop songs. Our system is seamlessly integrated into Hookpad, a web-based editor designed for the composition of lead sheets: symbolic music scores that describe melody and harmony. Hookpad Aria has numerous generation capabilities designed to assist users in non-sequential composition workflows, including: (1) generating left-to-right continuations of existing material, (2) filling in missing spans in the middle of existing material, and (3) generating harmony from melody and vice versa. Hookpad Aria is also a scalable data flywheel for music co-creation -- since its release in March 2024, Aria has generated 318k suggestions for 3k users who have accepted 74k into their songs. More information about Hookpad Aria is available at https://www.hooktheory.com/hookpad/aria

Problem

• Assists in writing Western pop songs
• Supports non-sequential composition workflows
• Generates harmony from melody and vice versa

Innovation

• Generative AI for songwriting
• Non-sequential composition workflows
• Scalable music co-creation data