RGB: RL Guided Whole-Body MPPI for Humanoid Control

2026-06-23

AI Summary

This work addresses the lack of robust and precise whole-body control methods for humanoid robots operating in complex contact-rich environments, as well as the limited flexibility of existing reinforcement learning (RL) policies in incorporating new task objectives online. The authors propose an RL-guided Model Predictive Path Integral (MPPI) framework that, for the first time, leverages a pretrained RL policy as a dynamically feasible sampling prior. By employing modular cost terms, the approach enables online adjustment of behavior to satisfy diverse task requirements without retraining. Evaluated on the 29-degree-of-freedom Unitree G1 robot in MuJoCo, the method achieves high-frequency closed-loop control at an average rate of 280 Hz. Under identical command interfaces, it significantly outperforms pure RL baselines by effectively correcting walking drift, accurately tracking additional whole-body reference signals, and seamlessly integrating new objectives in real time.

Abstract

Humanoid robots require whole-body controllers that are both robust and precise in contact-rich environments. While deep reinforcement learning (RL) achieves robust stability, its behavior is tightly coupled to the training objective and command interface, making it difficult to add new feedback objectives without retraining. In this study, we propose an RL guided whole-body model predictive path integral (MPPI) framework that acts as an add-on feedback controller on top of a pretrained RL policy. Instead of using RL policy as the final controller, we use it as a sampling prior that biases MPPI rollouts toward dynamically feasible behaviors. Task objectives are specified through modular MPPI cost terms, and MPPI closes the loop by continuously correcting the RL prior online to satisfy these objectives without retraining the policy. Simulations on a 29-DoF Unitree G1 humanoid in MuJoCo demonstrate stable high-rate control (average 280~Hz). The proposed method improves task-level precision over a pure RL baseline under the same command interface. This is achieved by correcting systematic drift during straight walking and tracking additional whole-body reference signals imposed through the cost.

Problem

• humanoid control
• whole-body control
• reinforcement learning
• feedback objectives
• contact-rich environments

Innovation

• Reinforcement Learning
• Model Predictive Path Integral
• Whole-Body Control
• Humanoid Robotics
• Online Feedback Correction