Learning Quadrupedal Locomotion for a Heavy Hydraulic Robot Using an Actuator Model

2025-12-01

IEEE Robotics and Automation Letters

AI Summary

This work addresses the sim-to-real transfer challenge in large-scale hydraulic quadrupedal robots, which arises from complex fluid dynamics and actuator hysteresis. To overcome this, the authors propose an analytical hydraulic actuator model that efficiently emulates joint torque dynamics within a reinforcement learning framework. This approach enables, for the first time, stable and robust command-tracking locomotion policies to be successfully transferred to heavy-duty hydraulic quadrupeds weighing over 300 kg, effectively breaking through the bottlenecks of actuator modeling and sim-to-real transfer under data-scarce conditions. Experimental results demonstrate that the learned policies exhibit excellent performance on the physical robot, with the proposed actuator model achieving inference times below one microsecond—significantly outperforming data-driven alternatives in both speed and fidelity.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

The simulation-to-reality (sim-to-real) transfer of large-scale hydraulic robots presents a significant challenge in robotics because of the inherent slow control response and complex fluid dynamics. The complex dynamics result from the multiple interconnected cylinder structure and the difference in fluid rates of the cylinders. These characteristics complicate detailed simulation for all joints, making it unsuitable for reinforcement learning (RL) applications. In this work, we propose an analytical actuator model driven by hydraulic dynamics to represent the complicated actuators. The model predicts joint torques for all 12 actuators in under 1 $\mu$s, allowing rapid processing in RL environments. We compare our model with neural network-based actuator models and demonstrate the advantages of our model in data-limited scenarios. The locomotion policy trained in RL with our model is deployed on a hydraulic quadruped robot, BeTheX-Q, which is over 300 kg. This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with RL on a heavy hydraulic quadruped robot, demonstrating advanced sim-to-real transferability.

Problem

• sim-to-real transfer
• hydraulic quadruped robot
• actuator modeling
• reinforcement learning
• locomotion control

Innovation

• hydraulic actuator model
• sim-to-real transfer
• reinforcement learning
• quadrupedal locomotion
• heavy robot