🤖 AI Summary
This study addresses the challenge of customizing orchestral accompaniments for pianists’ individual performance styles in the absence of digital scores, while mitigating spectral mismatches caused by inter-recording voicing differences. The authors propose a Dense-Sparse Dynamic Time Warping (DTW) approach that enhances alignment robustness under spectral mismatch by focusing on a sparse subset of audio frames containing salient temporal cues. Coupled with time-scale modification and multi-source audio alignment techniques, the method synchronizes the orchestral part to the user’s piano performance without requiring symbolic scores or source separation. Evaluated on a benchmark comprising four piano concerto movements, the proposed approach achieves performance comparable to or better than sophisticated baselines relying on source separation and spectral subtraction for accompaniment customization.
📝 Abstract
In this study, we explore how pianists can customize Music Minus One (MMO) concerto accompaniments to match their playing style. Bypassing the need for a symbolic score, often not available digitally, we use three types of audio data: solo piano recordings, MMO orchestra-only recordings, and mixed recordings of both piano and orchestra (e.g., from YouTube). The mixed recording serves as an intermediary reference to align the solo and orchestra parts, with only the orchestral part being adjusted through time-scale modification to synchronize with the user's playing. The main challenge with estimating these alignments is the spectral mismatch between recordings containing different musical parts. Motivated by this application scenario, we introduce Dense-Sparse DTW, a variant of Dynamic Time Warping (DTW) that is designed to improve robustness of alignments to spectral mismatch by focusing on aligning a selected subset of audio frames containing prominent timing cues. We collect and annotate data from four piano concerto movements and establish a framework for generating and evaluating customized accompaniment recordings. On this benchmark, we show that Dense-Sparse DTW has better or comparable performance than more complex approaches based on source separation and spectral subtraction techniques.