Booster Gym: An End-to-End Reinforcement Learning Framework for Humanoid Robot Locomotion

2025-06-18

AI Summary

Sim-to-real transfer of reinforcement learning (RL) policies for humanoid robots faces challenges including structural mismatch, poor disturbance robustness, and limited adaptability to unstructured terrains. This paper proposes an end-to-end integrated RL deployment framework that unifies physics-based modeling, domain randomization, hierarchical reward shaping, and real-time low-latency control interfacing—enabling plug-and-play deployment on physical platforms. Built upon PyTorch and Isaac Gym and employing the PPO algorithm, the framework jointly optimizes structural alignment, disturbance rejection (withstanding >50 N impulsive forces), and adaptive locomotion over challenging unstructured terrains (e.g., inclined slopes and gravel). Evaluated on the Booster T1 humanoid robot, the policy achieves zero-shot sim-to-real transfer, demonstrating omnidirectional walking and dynamic recovery without fine-tuning. The approach substantially narrows the reality gap and establishes a reproducible, scalable paradigm for RL-based humanoid locomotion control.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advancements in reinforcement learning (RL) have led to significant progress in humanoid robot locomotion, simplifying the design and training of motion policies in simulation. However, the numerous implementation details make transferring these policies to real-world robots a challenging task. To address this, we have developed a comprehensive code framework that covers the entire process from training to deployment, incorporating common RL training methods, domain randomization, reward function design, and solutions for handling parallel structures. This library is made available as a community resource, with detailed descriptions of its design and experimental results. We validate the framework on the Booster T1 robot, demonstrating that the trained policies seamlessly transfer to the physical platform, enabling capabilities such as omnidirectional walking, disturbance resistance, and terrain adaptability. We hope this work provides a convenient tool for the robotics community, accelerating the development of humanoid robots. The code can be found in https://github.com/BoosterRobotics/booster_gym.

Problem

• Transferring RL policies to real-world humanoid robots
• Simplifying training and deployment of locomotion policies
• Enabling omnidirectional walking and terrain adaptability

Innovation

• End-to-end RL framework for humanoid locomotion
• Incorporates domain randomization and reward design
• Seamless policy transfer to physical robots