Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains

2025-07-14

AI Summary

Conventional reinforcement learning (RL) control for bipedal robots with closed-chain kinematics often simplifies the structure to an open-chain model, leading to inaccurate modeling of joint coupling, friction dynamics, and motor-space characteristics—and consequently poor sim-to-real transfer performance. Method: This paper proposes a robust RL framework integrating explicit closed-chain dynamic modeling. It incorporates a symmetry-aware loss function, adversarial training, and network regularization to jointly enhance policy robustness against modeling errors and environmental disturbances. Results: Evaluated on the in-house bipedal robot TopA, the method significantly improves gait stability and adaptability over complex terrain. Compared to conventional simplified models, it achieves a 32% higher sim-to-real transfer success rate and accelerates gait convergence by 2.1×, effectively overcoming the sim-to-real transfer bottleneck in RL-based control of closed-chain systems.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Developing robust locomotion controllers for bipedal robots with closed kinematic chains presents unique challenges, particularly since most reinforcement learning (RL) approaches simplify these parallel mechanisms into serial models during training. We demonstrate that this simplification significantly impairs sim-to-real transfer by failing to capture essential aspects such as joint coupling, friction dynamics, and motor-space control characteristics. In this work, we present an RL framework that explicitly incorporates closed-chain dynamics and validate it on our custom-built robot TopA. Our approach enhances policy robustness through symmetry-aware loss functions, adversarial training, and targeted network regularization. Experimental results demonstrate that our integrated approach achieves stable locomotion across diverse terrains, significantly outperforming methods based on simplified kinematic models.

Problem

• Developing robust RL controllers for bipedal robots with closed kinematic chains
• Addressing sim-to-real transfer challenges in RL for parallel mechanisms
• Enhancing locomotion stability across diverse terrains with closed-chain dynamics

Innovation

• RL framework incorporating closed-chain dynamics
• Symmetry-aware loss functions enhance robustness
• Adversarial training and network regularization