RAMBO: RL-augmented Model-based Optimal Control for Whole-body Loco-manipulation

2025-04-09

AI Summary

This work addresses key challenges in whole-body loco-manipulation for quadrupedal robots—namely, low force interaction accuracy, poor robustness under dynamic motion, and the tension between model uncertainty and real-time execution. We propose a synergistic control framework that fuses model-predictive feedforward control with reinforcement learning (RL)-based feedback residual compensation. Specifically, we formulate a simplified multi-body dynamics model and employ quadratic programming (QP) to generate reaction-force feedforward commands, while an end-to-end RL policy corrects modeling errors and environmental disturbances. The framework explicitly balances end-effector trajectory tracking accuracy and contact compliance. Our key contribution is the first tightly coupled architecture integrating analytical dynamics with data-driven feedback—overcoming the interpretability limitations of purely learning-based force control and the poor generalizability of purely model-based approaches. Experiments on a physical quadruped robot demonstrate successful execution of diverse tasks—including cart pushing, tray balancing, and soft-object grasping—with significantly improved force control precision and motion robustness over end-to-end baselines.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Loco-manipulation -- coordinated locomotion and physical interaction with objects -- remains a major challenge for legged robots due to the need for both accurate force interaction and robustness to unmodeled dynamics. While model-based controllers provide interpretable dynamics-level planning and optimization, they are limited by model inaccuracies and computational cost. In contrast, learning-based methods offer robustness while struggling with precise modulation of interaction forces. We introduce RAMBO -- RL-Augmented Model-Based Optimal Control -- a hybrid framework that integrates model-based reaction force optimization using a simplified dynamics model and a feedback policy trained with reinforcement learning. The model-based module generates feedforward torques by solving a quadratic program, while the policy provides feedback residuals to enhance robustness in control execution. We validate our framework on a quadruped robot across a diverse set of real-world loco-manipulation tasks -- such as pushing a shopping cart, balancing a plate, and holding soft objects -- in both quadrupedal and bipedal walking. Our experiments demonstrate that RAMBO enables precise manipulation while achieving robust and dynamic locomotion, surpassing the performance of policies trained with end-to-end scheme. In addition, our method enables flexible trade-off between end-effector tracking accuracy with compliance.

Problem

• Achieving precise force interaction in legged robot loco-manipulation
• Balancing model-based control accuracy and computational efficiency
• Enhancing robustness in dynamic locomotion and object manipulation

Innovation

• Hybrid RL and model-based optimal control
• Feedforward torque via quadratic programming
• Feedback policy enhances robustness