Training Directional Locomotion for Quadrupedal Low-Cost Robotic Systems via Deep Reinforcement Learning

2025-03-14

AI Summary

To address the challenge of achieving robust directional locomotion—forward progression, turning, and complex trajectory tracking—with low-cost quadruped robots in real-world environments, this paper proposes an end-to-end control framework based on Proximal Policy Optimization (PPO). The key contribution is a novel yaw-offset–based heading randomization reset mechanism, introduced for the first time: during simulation training, target headings are dynamically sampled to encourage the policy to naturally acquire both high-frequency bidirectional turning and stable long-distance forward walking, significantly enhancing generalization and deployment robustness. Integrated with domain randomization, real-time physics-in-the-loop training, and embedded-system optimization, the method achieves full-stack deployment on a custom-built low-cost hardware platform. Experimental results demonstrate 100% success rates across multiple complex trajectory-tracking tasks—substantially outperforming a baseline capable only of forward walking—while markedly reducing manual tuning and human intervention.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this work we present Deep Reinforcement Learning (DRL) training of directional locomotion for low-cost quadrupedal robots in the real world. In particular, we exploit randomization of heading that the robot must follow to foster exploration of action-state transitions most useful for learning both forward locomotion as well as course adjustments. Changing the heading in episode resets to current yaw plus a random value drawn from a normal distribution yields policies able to follow complex trajectories involving frequent turns in both directions as well as long straight-line stretches. By repeatedly changing the heading, this method keeps the robot moving within the training platform and thus reduces human involvement and need for manual resets during the training. Real world experiments on a custom-built, low-cost quadruped demonstrate the efficacy of our method with the robot successfully navigating all validation tests. When trained with other approaches, the robot only succeeds in forward locomotion test and fails when turning is required.

Problem

• Training low-cost quadrupedal robots for directional locomotion using DRL.
• Enhancing robot's ability to follow complex trajectories with frequent turns.
• Reducing human involvement and manual resets during robot training.

Innovation

• Deep Reinforcement Learning for robot locomotion
• Randomized heading for enhanced exploration
• Low-cost quadrupedal robot navigation success