Explosive Jumping with Rigid and Articulated Soft Quadrupeds via Example Guided Reinforcement Learning

2025-03-20

AI Summary

This work addresses robust jumping control for quadrupedal robots equipped with passively compliant mechanical structures. Methodologically, we propose a transfer reinforcement learning framework integrating model-optimized imitation learning and reward-free fine-tuning. Leveraging a single demonstration trajectory, the approach combines proprioceptive-only sensing, parallel-elastic-actuator modeling, model predictive control, and progressive deep reinforcement learning to achieve controllable, explosive jumps under rigid–elastic hybrid configurations. Our key contributions include the first empirical validation that parallel elastic structures significantly enhance robustness: reducing landing position error by 11.1%, energy consumption by 15.2%, and peak joint torque by 15.8%. Moreover, the policy generalizes to multi-directional distance jumps and unknown ground irregularities up to 4 cm in height—relying solely on proprioception, without external sensing or handcrafted reward functions.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Achieving controlled jumping behaviour for a quadruped robot is a challenging task, especially when introducing passive compliance in mechanical design. This study addresses this challenge via imitation-based deep reinforcement learning with a progressive training process. To start, we learn the jumping skill by mimicking a coarse jumping example generated by model-based trajectory optimization. Subsequently, we generalize the learned policy to broader situations, including various distances in both forward and lateral directions, and then pursue robust jumping in unknown ground unevenness. In addition, without tuning the reward much, we learn the jumping policy for a quadruped with parallel elasticity. Results show that using the proposed method, i) the robot learns versatile jumps by learning only from a single demonstration, ii) the robot with parallel compliance reduces the landing error by 11.1%, saves energy cost by 15.2% and reduces the peak torque by 15.8%, compared to the rigid robot without parallel elasticity, iii) the robot can perform jumps of variable distances with robustness against ground unevenness (maximal 4cm height perturbations) using only proprioceptive perception.

Problem

• Achieving controlled jumping in quadruped robots with passive compliance.
• Generalizing jumping policies for varied distances and uneven terrains.
• Improving energy efficiency and reducing landing errors in compliant robots.

Innovation

• Imitation-based deep reinforcement learning for jumping
• Progressive training with model-based trajectory optimization
• Robust jumping on uneven ground using proprioception