CPR: Combining global composing, local performing and full-sequence refining in piano rendering with continuous autoregressive modelling

2026-09-16

AI Summary

本文提出CPR框架，通过结合全局构作、局部演奏和全序列优化，并采用连续自回归模型来解决MIDI到音乐渲染中音色再现与细节保留的问题。

Abstract

Prompt-conditioned piano MIDI-to-Music rendering aims to faithfully render target notes while reproducing the timbre of a reference recording. Existing approaches primarily follow two paradigms: autoregressive (AR) modeling and flow matching (or diffusion). Discrete-codec AR models provide causal temporal modeling, but quantization can discard acoustic detail. Flow matching better preserves acoustic structure in the cost of full-sequence attention costs and worse semantic structure. Continuous autoregressive models operate directly on continuous representations. It not only combines the condition-following ability of AR models and distribution-modeling capacity of flow matching but also bypasses the quantization bottleneck with lower computational costs. Building on this principle, we present Composer--Performer--Refiner (CPR) framework. Composer autoregressively predicts continuous hidden states, Performer generates 24kHz acoustic latents through local flow matching and Refiner then upsamples the waveform to 48 kHz. We further introduce Bottlenecked Representation Alignment (BREPA) and Modality--Time RoPE (MT-RoPE) to strengthen musical semantic structure in Composer hidden states and temporal alignments across modalities. Codes are available at https://github.com/FEAfeatherTHER/CPR_official

Problem

• piano rendering
• autoregressive modeling
• flow matching
• acoustic detail
• semantic structure

Innovation

• continuous autoregressive modeling
• Composer-Performer-Refiner (CPR)
• Bottlenecked Representation Alignment (BREPA)
• Modality-Time RoPE (MT-RoPE)