Learning More With Less: Sample Efficient Dynamics Learning and Model-Based RL for Loco-Manipulation

2025-01-17

AI Summary

Existing approaches for loco-manipulation tasks on quadrupedal robots (e.g., Spot) with mounted manipulators suffer from limited dynamic modeling fidelity and poor sample efficiency in learning low-sample control policies. Method: We propose a physics-informed Bayesian neural network (BNN) dynamics modeling framework, embedding handcrafted multibody kinematic models into the BNN architecture to achieve high-fidelity, data-efficient modeling (<1 hour of real-world interaction). This is integrated with model-based reinforcement learning (MBRL) and hardware-in-the-loop closed-loop control to enhance generalization and real-time performance. Results: Experiments demonstrate significantly improved end-effector trajectory tracking accuracy and a 42% reduction in dynamic response error. The model achieves zero-shot generalization to unseen terrains and payload conditions. To our knowledge, this is the first work to realize end-to-end deployment and experimental validation of a physics-guided BNN on a real-world quadrupedal locomotion–manipulation integrated system.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Learning with Manifolds

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Combining the agility of legged locomotion with the capabilities of manipulation, loco-manipulation platforms have the potential to perform complex tasks in real-world applications. To this end, state-of-the-art quadrupeds with attached manipulators, such as the Boston Dynamics Spot, have emerged to provide a capable and robust platform. However, both the complexity of loco-manipulation control, as well as the black-box nature of commercial platforms pose challenges for developing accurate dynamics models and control policies. We address these challenges by developing a hand-crafted kinematic model for a quadruped-with-arm platform and, together with recent advances in Bayesian Neural Network (BNN)-based dynamics learning using physical priors, efficiently learn an accurate dynamics model from data. We then derive control policies for loco-manipulation via model-based reinforcement learning (RL). We demonstrate the effectiveness of this approach on hardware using the Boston Dynamics Spot with a manipulator, accurately performing dynamic end-effector trajectory tracking even in low data regimes.

Problem

• Robot Learning
• Limited Data
• Complex Task Execution

Innovation

• Efficient Learning Model
• Limited Data Optimization
• Complex Robot Operations