Actor-Critic Cooperative Compensation to Model Predictive Control for Off-Road Autonomous Vehicles Under Unknown Dynamics

2025-03-01

AI Summary

This study addresses the low longitudinal velocity tracking accuracy and poor generalization of off-road unmanned vehicles on unknown deformable terrains (e.g., sand, gravel–soil mixtures, cohesive soils), caused by dynamics model mismatch. We propose an Actor–Critic-coordinated Model Predictive Control (MPC) framework. Its core innovation is a bidirectional predictive coupling mechanism between MPC and Actor–Critic: MPC ensures closed-loop robustness, while Actor–Critic online learns and compensates for modeling errors; both components achieve dynamic coordination via deep fusion of multi-source predictive information. The method significantly reduces reliance on labeled training data and enhances cross-terrain generalization. Experiments on diverse unknown deformable terrains demonstrate that the proposed approach reduces longitudinal velocity tracking error by 29.2% compared to conventional model-based MPC and by 10.2% compared to end-to-end learning methods.

Technology Category

• Intelligent Robots: State Estimation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This study presents an Actor-Critic Cooperative Compensated Model Predictive Controller (AC3MPC) designed to address unknown system dynamics. To avoid the difficulty of modeling highly complex dynamics and ensuring realtime control feasibility and performance, this work uses deep reinforcement learning with a model predictive controller in a cooperative framework to handle unknown dynamics. The model-based controller takes on the primary role as both controllers are provided with predictive information about the other. This improves tracking performance and retention of inherent robustness of the model predictive controller. We evaluate this framework for off-road autonomous driving on unknown deformable terrains that represent sandy deformable soil, sandy and rocky soil, and cohesive clay-like deformable soil. Our findings demonstrate that our controller statistically outperforms standalone model-based and learning-based controllers by upto 29.2% and 10.2%. This framework generalized well over varied and previously unseen terrain characteristics to track longitudinal reference speeds with lower errors. Furthermore, this required significantly less training data compared to purely learning-based controller, while delivering better performance even when under-trained.

Problem

• Addresses unknown dynamics in off-road autonomous vehicles
• Combines deep reinforcement learning with model predictive control
• Improves tracking performance on varied, deformable terrains

Innovation

• Actor-Critic Cooperative Compensated Model Predictive Controller
• Deep reinforcement learning with model predictive control
• Improved tracking performance on unknown terrains