Seconds-Aligned PCA-DAC Latent Diffusion for Symbolic-to-Audio Drum Rendering

2026-05-13

AI Summary

This work addresses the challenge of symbolic-controlled drum synthesis, which demands precise timing, dynamics, and acoustic plausibility. The authors propose Sec2Drum-DAC, a conditional latent diffusion model that, for the first time, integrates principal component analysis (PCA) with a frozen DAC latent space. Instead of directly generating waveforms, the model predicts PCA-compressed latent coordinates conditioned on physically time-aligned symbolic events. This approach yields a compact and continuous denoising target and enables deterministic reconstruction of high-dimensional latent representations. An auxiliary RVQ cross-entropy loss is introduced to further enhance performance. Evaluated on 1,733 test excerpts, the method outperforms baseline models in both spectral and transient fidelity metrics and achieves an optimal trade-off between quality and efficiency within 6–25 denoising steps.

Abstract

Symbolic-control drum generation requires preserving explicit event timing and dynamics while synthesizing acoustically plausible waveforms. We present Sec2Drum-DAC, a conditional latent-diffusion model for symbolic-to-audio drum rendering. The model conditions on event features sampled in physical time at codec-frame locations and predicts standardized principal-component coordinates of frozen DAC summed-codebook embeddings rather than waveform samples. In the evaluated DAC configuration, 72 principal components capture the observed training-frame summed-latent subspace under the stated SVD threshold, yielding a compact continuous denoising target with a deterministic reconstruction path to the 1024-dimensional DAC latent space before waveform decoding. Across 1,733 held-out four-beat windows, PCA diffusion improves paired spectral and transient metrics over deterministic PCA regression and a symbolic rendering baseline, while direct regression remains stronger on phase-sensitive waveform L1. Auxiliary RVQ cross-entropy improves short-step diffusion on mel error, onset-flux cosine, and waveform L1, with the most favorable trade-offs occurring at 6-25 denoising steps depending on the metric.

Problem

• symbolic-to-audio drum rendering
• event timing
• acoustic plausibility
• latent diffusion
• drum synthesis

Innovation

• latent diffusion
• PCA-based denoising
• symbolic-to-audio rendering
• DAC codec
• seconds-aligned conditioning