Opening the Design Space: Two Years of Performance with Intelligent Musical Instruments

2026-04-26

AI Summary

This work addresses the lack of artist-centered design in existing AI music tools, which often impedes their integration into creative practice. To bridge this gap, we present a low-cost, generative AI instrument platform built on single-board computers, leveraging MIDI interoperability and combining artist-curated datasets with lightweight local models to support longitudinal first-person artistic inquiry. We introduce an innovative interaction paradigm—“remapping over retraining”—and propose rapid input interleaving as a novel human-AI co-creation strategy. Five intelligent instrument prototypes were successfully developed, demonstrating that small-data models can serve as portable design resources. This approach lowers technical barriers while fostering inclusive and accessible AI-driven musical innovation.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Intelligent Robots: Embodied AI

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Machine generation of symbolic music and digital audio are hot topics but there have been relatively few digital musical instruments that integrate generative AI. Present musical AI tools are not artist centred and do not support experimentation or integrating into musical instruments or practices. This work introduces an inexpensive generative AI instrument platform based on a single board computer that connects via MIDI to other musical devices. The platform uses artist-collected datasets with models trained on a regular computer. This paper asks what the design space of intelligent musical instruments might look like when accessible and portable AI systems are available for artistic exploration. I contribute five examples of instruments created and tested through a two-year first-person artistic research process. These show that (re)mapping can replace retraining for discovering AI interaction, that fast input interleaving is a new co-creative strategy, that small-data AI models can be a transportable design resource, and that cheap hardware can lower barriers to inclusion. This work could enable artists to explore new interaction and performance schemes with intelligent musical instruments.

Problem

• intelligent musical instruments
• artist-centred design
• generative AI
• musical interaction
• creative exploration

Innovation

• generative AI
• intelligent musical instruments
• artist-centered design
• small-data models
• fast input interleaving