Coordinated Humanoid Robot Locomotion with Symmetry Equivariant Reinforcement Learning Policy

2025-08-02

AI Summary

Existing deep reinforcement learning methods neglect the morphological symmetry of humanoid robots, leading to uncoordinated locomotion and poor balance control. To address this, we propose SE-Policy—a novel policy optimization framework that rigorously enforces spatial symmetry: it embeds *equivariance* into the policy network to ensure coordinated bilateral actuation, and imposes *invariance* in the critic network to enhance training stability—both without introducing additional hyperparameters. Built upon symmetric equivariant neural networks, SE-Policy enables spatiotemporally consistent motion control across simulation and real-world deployment. Evaluated on velocity-tracking tasks in MuJoCo and on the Unitree G1 robot, SE-Policy achieves up to 40% higher tracking accuracy over state-of-the-art methods, while significantly improving motion coordination, robustness to disturbances, and generalization across diverse terrains and gaits.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The human nervous system exhibits bilateral symmetry, enabling coordinated and balanced movements. However, existing Deep Reinforcement Learning (DRL) methods for humanoid robots neglect morphological symmetry of the robot, leading to uncoordinated and suboptimal behaviors. Inspired by human motor control, we propose Symmetry Equivariant Policy (SE-Policy), a new DRL framework that embeds strict symmetry equivariance in the actor and symmetry invariance in the critic without additional hyperparameters. SE-Policy enforces consistent behaviors across symmetric observations, producing temporally and spatially coordinated motions with higher task performance. Extensive experiments on velocity tracking tasks, conducted in both simulation and real-world deployment with the Unitree G1 humanoid robot, demonstrate that SE-Policy improves tracking accuracy by up to 40% compared to state-of-the-art baselines, while achieving superior spatial-temporal coordination. These results demonstrate the effectiveness of SE-Policy and its broad applicability to humanoid robots.

Problem

• Improving humanoid robot locomotion coordination using symmetry equivariant reinforcement learning
• Addressing suboptimal behaviors in DRL by embedding morphological symmetry in policies
• Enhancing tracking accuracy and spatial-temporal coordination in humanoid robot movements

Innovation

• Symmetry Equivariant Policy for coordinated locomotion
• Embeds symmetry equivariance without extra hyperparameters
• Improves tracking accuracy by 40% in experiments