DFM: Deep Fourier Mimic for Expressive Dance Motion Learning

2025-02-16

AI Summary

Entertainment robots exhibit rigid, non-interactive dance motions due to reliance on pre-programmed, periodic motion templates lacking dynamic responsiveness. Method: We propose a natural dance motion learning framework for multi-task coordination. Departing from conventional periodic constraints, we introduce a flexible Fourier motion representation—enabling the first deep integration of frequency-domain modeling with Proximal Policy Optimization (PPO) reinforcement learning. We further design a joint deep Fourier encoder–multi-task controller architecture that jointly optimizes dance generation, bipedal locomotion, and gaze control in both time and frequency domains. Contribution/Results: Compared to scripted playback baselines, our approach significantly improves motion tracking accuracy and expressive fidelity. It enables high-fidelity, real-time dance generation with synchronized locomotion and gaze control on physical robots—establishing a novel paradigm for modeling natural, embodied dynamic behaviors in intelligent agents.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

As entertainment robots gain popularity, the demand for natural and expressive motion, particularly in dancing, continues to rise. Traditionally, dancing motions have been manually designed by artists, a process that is both labor-intensive and restricted to simple motion playback, lacking the flexibility to incorporate additional tasks such as locomotion or gaze control during dancing. To overcome these challenges, we introduce Deep Fourier Mimic (DFM), a novel method that combines advanced motion representation with Reinforcement Learning (RL) to enable smooth transitions between motions while concurrently managing auxiliary tasks during dance sequences. While previous frequency domain based motion representations have successfully encoded dance motions into latent parameters, they often impose overly rigid periodic assumptions at the local level, resulting in reduced tracking accuracy and motion expressiveness, which is a critical aspect for entertainment robots. By relaxing these locally periodic constraints, our approach not only enhances tracking precision but also facilitates smooth transitions between different motions. Furthermore, the learned RL policy that supports simultaneous base activities, such as locomotion and gaze control, allows entertainment robots to engage more dynamically and interactively with users rather than merely replaying static, pre-designed dance routines.

Problem

• Enhance dance motion expressiveness in robots
• Enable smooth transitions between dance motions
• Support auxiliary tasks during robot dancing

Innovation

• Deep Fourier Mimic method
• Reinforcement Learning integration
• Enhanced motion expressiveness