ExBody2: Advanced Expressive Humanoid Whole-Body Control

2024-12-17

arXiv.org

AI Summary

This work addresses the challenge of achieving stable and robust whole-body control for embodied humanoid robots executing dynamic, highly expressive motions—such as walking, squatting, and dancing—in real-world environments. We propose a novel paradigm that decouples full-body velocity planning from keypoint tracking. Our method employs a two-stage teacher–student framework: the teacher policy generates and filters intermediate motion data based on kinematic feasibility to enhance sim-to-real transfer efficiency; the student controller integrates motion-capture priors with simulation-based training to enable end-to-end whole-body control, and supports adjustable trade-offs between generalizability and task-specific accuracy via few-shot fine-tuning. The approach is successfully deployed on physical robots, demonstrating significant improvements in motion tracking fidelity. Results validate a synergistic optimization mechanism balancing control performance and generalization capability.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper tackles the challenge of enabling real-world humanoid robots to perform expressive and dynamic whole-body motions while maintaining overall stability and robustness. We propose Advanced Expressive Whole-Body Control (Exbody2), a method for producing whole-body tracking controllers that are trained on both human motion capture and simulated data and then transferred to the real world. We introduce a technique for decoupling the velocity tracking of the entire body from tracking body landmarks. We use a teacher policy to produce intermediate data that better conforms to the robot's kinematics and to automatically filter away infeasible whole-body motions. This two-step approach enabled us to produce a student policy that can be deployed on the robot that can walk, crouch, and dance. We also provide insight into the trade-off between versatility and the tracking performance on specific motions. We observed significant improvement of tracking performance after fine-tuning on a small amount of data, at the expense of the others.

Problem

• Enable humanoid robots to perform expressive, dynamic whole-body motions.
• Develop a method for real-world transfer of whole-body control.
• Balance versatility and specific motion tracking performance.

Innovation

• Combines human motion capture with simulated data
• Decouples velocity tracking from body landmarks
• Uses teacher policy for feasible motion filtering