Re4MPC: Reactive Nonlinear MPC for Multi-model Motion Planning via Deep Reinforcement Learning

2025-06-10

AI Summary

To address the high computational cost of conventional nonlinear model predictive control (NMPC) in real-time motion planning for high-degree-of-freedom robots (e.g., mobile manipulators), this paper proposes a deep reinforcement learning (DRL)-driven reactive NMPC framework. The method reformulates NMPC as a learnable reactive decision-making problem, establishing a mathematically rigorous co-optimization framework that jointly adapts the model structure, cost function, and constraint set in real time. Leveraging multi-model dynamics representation and online trajectory optimization, the approach achieves a 32% improvement in end-effector goal attainment success rate over a full-body NMPC baseline in simulation, while reducing average planning latency by 5.8×. This demonstrates a significant advance in balancing real-time performance with motion accuracy for complex robotic systems.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Traditional motion planning methods for robots with many degrees-of-freedom, such as mobile manipulators, are often computationally prohibitive for real-world settings. In this paper, we propose a novel multi-model motion planning pipeline, termed Re4MPC, which computes trajectories using Nonlinear Model Predictive Control (NMPC). Re4MPC generates trajectories in a computationally efficient manner by reactively selecting the model, cost, and constraints of the NMPC problem depending on the complexity of the task and robot state. The policy for this reactive decision-making is learned via a Deep Reinforcement Learning (DRL) framework. We introduce a mathematical formulation to integrate NMPC into this DRL framework. To validate our methodology and design choices, we evaluate DRL training and test outcomes in a physics-based simulation involving a mobile manipulator. Experimental results demonstrate that Re4MPC is more computationally efficient and achieves higher success rates in reaching end-effector goals than the NMPC baseline, which computes whole-body trajectories without our learning mechanism.

Problem

• Efficient motion planning for high-DOF robots
• Reactive model selection for NMPC via DRL
• Improving computational efficiency and success rates

Innovation

• Reactive NMPC for efficient multi-model planning
• DRL-based policy for adaptive model selection
• Integrated NMPC-DRL framework for robotics