Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning

2026-07-02

AI Summary

This work addresses the sim-to-real transfer failure commonly encountered in reinforcement learning due to mismatches between idealized actuator models used in simulation and the nonlinear, hardware-dependent motor dynamics of real robots. To bridge this gap, the authors propose “actuator reality shaping,” a method that deploys a two-degree-of-freedom feedforward–feedback controller on physical hardware to shape the closed-loop actuator response to closely match an ideal second-order reference model assumed in simulation. Notably, this approach requires no system identification or learned actuator models and enables zero-shot policy deployment through a standardized actuator interface. Experiments across diverse platforms—including single-joint servos, a 7-DoF manipulator, wheeled-legged robots, and humanoids—demonstrate substantial reductions in tracking error and successful zero-shot transfer across multiple tasks and systems, thereby shifting the paradigm from increasing simulation fidelity to unifying real-world actuator behavior to conform to simulation assumptions.

Abstract

Sim-to-real transfer in robot learning is often limited by discrepancies between the ideal actuator dynamics assumed during policy training and the nonlinear, hardware-dependent behavior of physical motors. While conventional approaches attempt to bridge this gap by increasing simulator fidelity through system identification, domain randomization, or learned actuator models, we introduce an alternative paradigm: actuator reality shaping. Instead of modifying the simulator to match the real world, our method shapes the closed-loop behavior of physical actuators to match the idealized second-order reference dynamics used in simulation. By equipping each joint with a two-degree-of-freedom feedforward--feedback controller, we decouple reference-response shaping from robust stabilization, thereby providing a standardized actuator interface for reinforcement learning policies. As a result, policies trained only with the prescribed reference model can be deployed zero-shot on real hardware without task-level fine-tuning or learned actuator models. We validate the approach on a single-joint high-gear-ratio servo under external loads and a 7-DOF robotic arm reaching task, where actuator reality shaping substantially reduces sim-to-real tracking error and improves zero-shot task performance compared with standard servo-control and representative real-to-sim-to-real baselines. We further demonstrate zero-shot transfer on a wheeled-legged robot driving over a slope and a humanoid robot walking, suggesting that actuator reality shaping can serve as a reusable interface for robot learning across diverse hardware platforms.

Problem

• sim-to-real transfer
• actuator dynamics
• zero-shot learning
• robot learning
• hardware discrepancy

Innovation

• actuator reality shaping
• zero-shot sim-to-real
• reference dynamics tracking
• two-degree-of-freedom control
• standardized actuator interface