Gait-Conditioned Reinforcement Learning with Multi-Phase Curriculum for Humanoid Locomotion

2025-05-27

AI Summary

This work addresses the challenge of unifying multi-gait control for humanoid robots. We propose a homogeneous cyclic policy framework based on reinforcement learning (RL). Methodologically, we design a gait-encoding-driven dynamic reward routing mechanism and biologically inspired reward functions—such as straight-knee support and arm-leg coordination—integrated with an RNN-based policy network conditioned on one-hot gait IDs and trained via multi-stage progressive curriculum learning, eliminating reliance on motion-capture data or reference trajectories. Our key contributions are: (i) the first demonstration of cross-modal coordinated control—encompassing standing, walking, running, and smooth gait transitions—within a single RL policy; (ii) robust full-gait performance in simulation; and (iii) real-world validation on the Unitree G1 robot, confirming stability and motion coordination during standing, walking, and walk-stop transitions, significantly enhancing movement naturalness and generalization capability.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We present a unified gait-conditioned reinforcement learning framework that enables humanoid robots to perform standing, walking, running, and smooth transitions within a single recurrent policy. A compact reward routing mechanism dynamically activates gait-specific objectives based on a one-hot gait ID, mitigating reward interference and supporting stable multi-gait learning. Human-inspired reward terms promote biomechanically natural motions, such as straight-knee stance and coordinated arm-leg swing, without requiring motion capture data. A structured curriculum progressively introduces gait complexity and expands command space over multiple phases. In simulation, the policy successfully achieves robust standing, walking, running, and gait transitions. On the real Unitree G1 humanoid, we validate standing, walking, and walk-to-stand transitions, demonstrating stable and coordinated locomotion. This work provides a scalable, reference-free solution toward versatile and naturalistic humanoid control across diverse modes and environments.

Problem

• Enables humanoid robots to perform diverse locomotion modes
• Mitigates reward interference in multi-gait learning
• Promotes biomechanically natural motions without motion capture

Innovation

• Gait-conditioned reinforcement learning for humanoid locomotion
• Dynamic reward routing with gait-specific objectives
• Multi-phase curriculum for progressive gait complexity