Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation

2026-07-10

AI Summary

Existing methods struggle to generate minute-long, high-resolution dance videos synchronized with music, often hindered by the temporal limitations of diffusion models, which lead to temporal drift, identity inconsistency, and repetitive motions. This work proposes a hierarchical generation framework that decouples music-to-dance synthesis into global keyframe planning and local temporal refinement, leveraging full-song audio context to ensure long-term coherence while supporting dual conditioning on both audio and text. The approach introduces a novel time-mapping RoPE embedding with dynamic frame-rate adaptation for precise audio-motion alignment, incorporates an optical flow loss to enhance motion continuity, and integrates motion velocity control to preserve fine details of fast movements. To our knowledge, this is the first method capable of stably generating high-fidelity dance videos exceeding one minute in duration at 720p resolution and 30 fps, achieving state-of-the-art performance across five distinct dance styles.

Abstract

Generating long-duration, high-definition, and rhythmically synchronized dance videos directly from music remains a significant challenge, primarily due to the temporal constraints of current diffusion models, which typically fail beyond 20 seconds. Existing approaches, whether they rely on intermediate 3D skeletons or on end-to-end video synthesis, suffer from temporal drift, identity inconsistency, and repetitive motion patterns when extended to longer horizons. To address these limitations, we propose a novel hierarchical framework for minute-scale coherent music-to-dance generation. Our method decouples the process into global keyframe planning and local temporal refinement, leveraging full-track musical context to ensure long-range coherence. Key innovations include dynamic frame rate adaptation via time-mapped RoPE embeddings for precise alignment, an optical-flow-based loss function to enhance motion continuity, and motion-speed control to preserve high-fidelity details during rapid movements. Extensive experiments demonstrate that our framework surpasses the conventional duration barrier, generating stable, 720p/30fps videos exceeding one minute with superior temporal stability. Furthermore, the model exhibits robust versatility across five distinct dance genres, conditioned on both audio and textual prompts, establishing a new state-of-the-art in coherent, long-form dance video synthesis.

Problem

• music-to-dance generation
• long-duration video synthesis
• temporal coherence
• rhythmic synchronization
• motion consistency

Innovation

• hierarchical framework
• dynamic frame rate adaptation
• optical-flow-based loss
• motion-speed control
• long-form dance synthesis