Sim-to-Real of Humanoid Locomotion Policies via Joint Torque Space Perturbation Injection

2026-03-23

AI Summary

This work addresses the challenge of policy failure in sim-to-real transfer for humanoid robots caused by model mismatch and unmodeled dynamics. To this end, the authors propose a state-dependent, non-parametric perturbation method that leverages neural networks to generate system-state-aware disturbances in joint torque space. Unlike conventional domain randomization based on fixed parameters, this approach more faithfully captures complex real-world uncertainties—such as nonlinear actuator dynamics and contact compliance—by adapting perturbations to the current state of the system. When integrated with reinforcement learning–trained locomotion policies, the method yields significant robustness against unseen disturbances, demonstrating strong performance both in simulation and on physical hardware, thereby substantially improving sim-to-real transfer fidelity.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

This paper proposes a novel alternative to existing sim-to-real methods for training control policies with simulated experiences. Unlike prior methods that typically rely on domain randomization over a fixed finite set of parameters, the proposed approach injects state-dependent perturbations into the input joint torque during forward simulation. These perturbations are designed to simulate a broader spectrum of reality gaps than standard parameter randomization without requiring additional training. By using neural networks as flexible perturbation generators, the proposed method can represent complex, state-dependent uncertainties, such as nonlinear actuator dynamics and contact compliance, that parametric randomization cannot capture. Experimental results demonstrate that the proposed approach enables humanoid locomotion policies to achieve superior robustness against complex, unseen reality gaps in both simulation and real-world deployment.

Problem

• sim-to-real
• humanoid locomotion
• reality gap
• robustness
• control policies

Innovation

• sim-to-real
• joint torque perturbation
• state-dependent uncertainty
• domain randomization
• humanoid locomotion