Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control

2026-01-07

arXiv.org

AI Summary

This work addresses the reproducibility challenges in long-form song generation research, which has been hindered by the absence of publicly available datasets, impeding fair comparison and technical progress. We propose an open-source, end-to-end controllable system for long-form song synthesis that leverages the Qwen language model integrated with MuCodec discrete audio tokens, enabling fine-grained style control through single-stage supervised fine-tuning. To support this effort, we release the first open dataset comprising 116,000 licensed synthetic songs, accompanied by a complete pipeline for training, evaluation, and deployment. Experimental results demonstrate superior performance in phoneme error rate, text-to-music style consistency, and audio aesthetic quality, while enabling controllable generation at the segment level of musical structure, thereby significantly enhancing reproducibility in this research domain.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Recent commercial systems such as Suno demonstrate strong capabilities in long-form song generation, while academic research remains largely non-reproducible due to the lack of publicly available training data, hindering fair comparison and progress. To this end, we release a fully open-source system for long-form song generation with fine-grained style conditioning, including a licensed synthetic dataset, training and evaluation pipelines, and Muse, an easy-to-deploy song generation model. The dataset consists of 116k fully licensed synthetic songs with automatically generated lyrics and style descriptions paired with audio synthesized by SunoV5. We train Muse via single-stage supervised finetuning of a Qwen-based language model extended with discrete audio tokens using MuCodec, without task-specific losses, auxiliary objectives, or additional architectural components. Our evaluations find that although Muse is trained with a modest data scale and model size, it achieves competitive performance on phoneme error rate, text--music style similarity, and audio aesthetic quality, while enabling controllable segment-level generation across different musical structures. All data, model weights, and training and evaluation pipelines will be publicly released, paving the way for continued progress in controllable long-form song generation research. The project repository is available at https://github.com/yuhui1038/Muse.

Problem

• reproducibility
• long-form song generation
• fine-grained style control
• training data availability
• controllable music generation

Innovation

• fine-grained style control
• reproducible song generation
• synthetic music dataset
• single-stage finetuning
• discrete audio tokens