Music-to-Dance Generation via Atomic Movements

2026-07-15

AI Summary

Existing music-driven dance generation methods often overlook the compositional structure of movement, resulting in outputs that lack structural coherence and controllability. This work proposes modeling dance as a sequence of semantically interpretable atomic actions and introduces a two-stage generation framework: first planning the type, duration, and timing of atomic actions based on input music, then synthesizing them into smooth, coherent full-body motion. A reusable and editable motion vocabulary is constructed through large-scale motion segmentation, clustering, and semantic relabeling via large language models, enabling structure-aware dance synthesis. Experiments demonstrate that the proposed approach significantly outperforms existing methods in structural coherence, rhythmic alignment, and perceptual naturalness, while supporting flexible editing through its explicit structural representation.

Abstract

Music-driven dance generation aims to produce human motion that is both rhythmically synchronized and semantically consistent with music. While recent neural approaches have achieved impressive visual realism, they typically model motion as a continuous signal and neglect its compositional nature, making generated dances structurally incoherent and difficult to control. In this work, we introduce a structure-aware framework that models choreography as a sequence of atomic movements-semantically interpretable motion events that serve as the building blocks of dance. To construct this atomic movement vocabulary, we first segment large-scale dance data and cluster them into atomic movement groups. We then employ a large language model to semantically relabel and refine the clusters, yielding a set of interpretable and reusable atomic movements. Based on these atomic movement annotations, we design a two-stage generation framework that mirrors the human choreography process. In the atomic movement planning stage, the model predicts the type, duration, and timing of atomic movements conditioned on the input music, forming a symbolic dance allocation. In the completion stage, a transition-aware generator synthesizes smooth and stylistically coherent motion conditioned on the planned structure. Extensive experiments demonstrate that our method produces dances with significantly improved structural coherence, rhythmic alignment, and perceptual naturalness compared to existing baselines, while providing enhanced interpretability and controllable editing through explicit structural representation.

Problem

• music-to-dance generation
• structural coherence
• atomic movements
• motion compositionality
• dance controllability

Innovation

• atomic movements
• structure-aware dance generation
• music-driven motion synthesis
• semantic motion clustering
• two-stage choreography framework