MuScriptor: An Open Model for Multi-Instrument Music Transcription

2026-07-09

AI Summary

This work addresses the limited performance of existing automatic music transcription methods on real-world multi-instrument mixtures, which stems primarily from poor generalization of synthetic training data and the absence of explicit modeling of instrument co-occurrence. To overcome these challenges, the authors propose MuScriptor, a high-quality multi-instrument transcription model tailored for real-world scenarios. MuScriptor employs a three-stage training strategy: pretraining on synthetic data, fine-tuning on real musical recordings, and reinforcement learning–based post-training. Additionally, it incorporates an instrument presence conditioning mechanism that enables customizable, on-demand transcription outputs. The model has been validated across diverse musical genres and demonstrates strong empirical performance, with code and models publicly released to facilitate further research.

Abstract

Existing methods for automatic music transcription are often limited to single-instrument recordings or fail on complex, real music mixes. Although previous work utilizes synthetic training data, the resulting models generalize poorly, leading to largely unusable transcription output in realistic, multi-instrument settings. In this work, we analyze the effectiveness of synthetic data for pre-training while combining it with fine-tuning on real music audio and post-training using reinforcement learning. We further introduce conditioning on instrument presence to customize transcriptions. Finally, we release MuScriptor, an open-weight multi-instrument music transcription model that works on real-world music recordings from across a diverse range of musical genres.

Problem

• music transcription
• multi-instrument
• real-world audio
• model generalization
• synthetic data

Innovation

• multi-instrument music transcription
• synthetic data pre-training
• reinforcement learning post-training
• instrument conditioning
• open-weight model