MusicLayout: Explicit Structural Planning for Controllable Text-to-Music Generation

2026-08-09

AI Summary

Current text-to-music generation approaches rely on global textual prompts and lack explicit modeling and controllable planning of musical structure. This work proposes MusicLayout—an editable and interpretable intermediate representation that explicitly encodes time-aligned structural elements such as sections, texture, repetition, variation, and instrumentation. MusicLayout is integrated into a unified autoregressive framework: it is first generated as a structural plan and then used as a conditioning signal for audio token synthesis. This approach enables, for the first time, user intervention and control over musical structure prior to audio generation. Experimental results demonstrate that MusicLayout significantly enhances long-range structural consistency in generated music and supports flexible, layout-level editing and manipulation.

Abstract

Text-to-music generation has advanced rapidly, but current systems still rely primarily on global text prompts, leaving the structural organization of generated music implicit and difficult to inspect, control, or revise before audio generation. To address this issue, we introduce MusicLayout, an explicit intermediate representation for controlling musical structure in text-to-music generation. MusicLayout describes a musical piece as a time-aligned layout of sections, textures, repetitions, variations, and instrument-level arrangements, serving as an interpretable planning layer between textual intent and the generated music. We integrate MusicLayout into a text-to-music framework built on a unified autoregressive formulation, where the model first generates a MusicLayout representation and subsequently predicts audio tokens conditioned on this representation within a single sequence. The resulting MusicLayout can be inspected and modified prior to audio generation, providing a mechanism for layout-level structural control. We evaluate MusicLayout through layout-conditioned generation, layout manipulation experiments, and matched-data ablations, providing evidence that explicit layout planning can improve long-range structural organization and support layout-level control.

Problem

• text-to-music generation
• musical structure
• structural control
• explicit planning
• controllable generation

Innovation

• MusicLayout
• explicit structural planning
• controllable text-to-music generation
• intermediate representation
• autoregressive music generation