Harmonious Group Choreography with Trajectory-Controllable Diffusion

2024-03-10

arXiv.org

AI Summary

This work addresses two key challenges in music-driven group dance generation: inter-dancer collisions and intra-dancer foot sliding. To this end, we propose Trajectory-Controlled Diffusion (TCDiff), a novel diffusion-based framework. Methodologically, TCDiff introduces a first-of-its-kind Dance Trajectory Navigator and Step Adapter to explicitly enforce collision-free trajectories and foot stability during the denoising process. We further design a distance consistency loss and a relative forward kinematics loss to jointly optimize group-level spatial coordination and individual-level physical plausibility. Extensive experiments demonstrate that TCDiff significantly outperforms state-of-the-art methods across quantitative metrics—including collision rate, foot sliding error, music alignment accuracy—and qualitative user preference studies. To our knowledge, TCDiff is the first approach to achieve high spatiotemporal coordination, strong musical synchronization, zero collisions, and physically stable foot contact—enabling high-fidelity, realistic group dance synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Security and Privacy: Data transparency and provenance

Abstract

Creating group choreography from music is crucial in cultural entertainment and virtual reality, with a focus on generating harmonious movements. Despite growing interest, recent approaches often struggle with two major challenges: multi-dancer collisions and single-dancer foot sliding. To address these challenges, we propose a Trajectory-Controllable Diffusion (TCDiff) framework, which leverages non-overlapping trajectories to ensure coherent and aesthetically pleasing dance movements. To mitigate collisions, we introduce a Dance-Trajectory Navigator that generates collision-free trajectories for multiple dancers, utilizing a distance-consistency loss to maintain optimal spacing. Furthermore, to reduce foot sliding, we present a footwork adaptor that adjusts trajectory displacement between frames, supported by a relative forward-kinematic loss to further reinforce the correlation between movements and trajectories. Experiments demonstrate our method's superiority.

Problem

• Prevent multi-dancer collisions in choreography
• Eliminate single-dancer foot sliding issues
• Generate harmonious dance movements from music

Innovation

• Trajectory-Controllable Diffusion framework
• Dance-Trajectory Navigator for collision avoidance
• Footwork adaptor to minimize foot sliding