Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning

2025-04-19

AI Summary

Humanoid whole-body coordination faces challenges including poor stability, low imitation fidelity, and inefficient policy learning due to functional coupling between upper and lower limbs. This paper proposes a torso–leg adversarial policy learning framework: the lower body focuses on robust gait tracking (responding to velocity commands), while the upper body prioritizes high-fidelity motion reproduction; both are jointly optimized via an adversarial mechanism. The method is trained via reinforcement learning in MuJoCo and successfully deployed on the full-scale Unitree H1 humanoid robot. Key contributions include: (1) the first decoupled–adversarial whole-body control paradigm; (2) the first large-scale, high-quality whole-body motion control dataset designed for real-world deployment, publicly released; and (3) comprehensive validation—both in simulation and on hardware—of stable locomotion, precise motion tracking, and cross-task generalization, enabling complex teleoperated loco-manipulation applications.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Humans exhibit diverse and expressive whole-body movements. However, attaining human-like whole-body coordination in humanoid robots remains challenging, as conventional approaches that mimic whole-body motions often neglect the distinct roles of upper and lower body. This oversight leads to computationally intensive policy learning and frequently causes robot instability and falls during real-world execution. To address these issues, we propose Adversarial Locomotion and Motion Imitation (ALMI), a novel framework that enables adversarial policy learning between upper and lower body. Specifically, the lower body aims to provide robust locomotion capabilities to follow velocity commands while the upper body tracks various motions. Conversely, the upper-body policy ensures effective motion tracking when the robot executes velocity-based movements. Through iterative updates, these policies achieve coordinated whole-body control, which can be extended to loco-manipulation tasks with teleoperation systems. Extensive experiments demonstrate that our method achieves robust locomotion and precise motion tracking in both simulation and on the full-size Unitree H1 robot. Additionally, we release a large-scale whole-body motion control dataset featuring high-quality episodic trajectories from MuJoCo simulations deployable on real robots. The project page is https://almi-humanoid.github.io.

Problem

• Achieving human-like whole-body coordination in humanoid robots
• Addressing robot instability and falls during motion execution
• Enabling robust locomotion and precise motion tracking simultaneously

Innovation

• Adversarial policy learning for humanoid coordination
• Separate upper and lower body control policies
• Large-scale motion dataset for real robot deployment