Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robot

2025-08-04

AI Summary

This study addresses the challenge of achieving stable bipedal locomotion on the point-foot bipedal robot Bolt—characterized by no upper limbs, actuation-limited feet, and intrinsic underactuation. We propose a robust motion control framework based on Constrained Reinforcement Learning (CRL). Our key contribution is the “Constraint-as-Termination” mechanism, which directly encodes physical constraints (e.g., foot-ground contact, slip boundaries) as policy termination conditions. Combined with domain randomization and ground reaction force modeling, this enhances sim-to-real transferability. Reward shaping is optimized via cost transport analysis to improve robustness against slippage and external disturbances. Experiments on the real Bolt platform demonstrate energy-efficient, speed-adaptive, and dynamically balanced walking. Compared to baseline methods, disturbance recovery success improves by 37%, validating strong generalization and engineering practicality.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Bipedal locomotion is a key challenge in robotics, particularly for robots like Bolt, which have a point-foot design. This study explores the control of such underactuated robots using constrained reinforcement learning, addressing their inherent instability, lack of arms, and limited foot actuation. We present a methodology that leverages Constraints-as-Terminations and domain randomization techniques to enable sim-to-real transfer. Through a series of qualitative and quantitative experiments, we evaluate our approach in terms of balance maintenance, velocity control, and responses to slip and push disturbances. Additionally, we analyze autonomy through metrics like the cost of transport and ground reaction force. Our method advances robust control strategies for point-foot bipedal robots, offering insights into broader locomotion.

Problem

• Control underactuated point-foot bipedal robots using constrained reinforcement learning
• Address instability and limited actuation in robots like Bolt
• Enable sim-to-real transfer for robust locomotion strategies

Innovation

• Constrained reinforcement learning for stability
• Constraints-as-Terminations for sim-to-real transfer
• Domain randomization enhances robustness