Inverse RL Scene Dynamics Learning for Nonlinear Predictive Control in Autonomous Vehicles.

2025-03-27

IEEE Transactions on Neural Networks and Learning Systems

AI Summary

To address the poor modeling accuracy and limited adaptability of nonlinear model predictive control (NMPC) in dynamic autonomous driving scenarios, this paper proposes a deep learning–driven, scene-aware NMPC framework (DL-NMPC-SD). Methodologically, it innovatively embeds inverse reinforcement learning (IRL)–guided scene dynamics modeling into deep neural network layers, jointly integrating prior vehicle kinematics with a nonlinear state-space model learned from temporal ranging measurements. An enhanced memory module uniformly encodes heterogeneous time-series sensory inputs, while an improved deep Q-learning algorithm solves the Bellman-optimal trajectory planning problem. Evaluated on GridSim simulation, the RovisLab mobile platform, and real-world road tests, DL-NMPC-SD significantly outperforms baseline methods—including dynamic window approach (DWA), end-to-end learning, and standard reinforcement learning—in both trajectory tracking accuracy and robustness to complex, dynamic environments.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This article introduces the deep learning-based nonlinear model predictive controller with scene dynamics (DL-NMPC-SD) method for autonomous navigation. DL-NMPC-SD uses an a priori nominal vehicle model in combination with a scene dynamics model learned from temporal range sensing information. The scene dynamics model is responsible for estimating the desired vehicle trajectory, as well as to adjust the true system model used by the underlying model predictive controller. We propose to encode the scene dynamics model within the layers of a deep neural network, which acts as a nonlinear approximator for the high-order state space of the operating conditions. The model is learned based on temporal sequences of range-sensing observations and system states, both integrated by an Augmented Memory component. We use inverse reinforcement learning (IRL) and the Bellman optimality principle to train our learning controller with a modified version of the deep Q-learning (DQL) algorithm, enabling us to estimate the desired state trajectory as an optimal action-value function. We have evaluated DL-NMPC-SD against the baseline dynamic window approach (DWA), as well as against two state-of-the-art End2End and RL methods, respectively. The performance has been measured in three experiments: 1) in our GridSim virtual environment; 2) on indoor and outdoor navigation tasks using our RovisLab autonomous mobile test unit (AMTU) platform; and 3) on a full-scale autonomous test vehicle driving on public roads.

Problem

• Develops DL-based nonlinear predictive control for autonomous vehicles
• Learns scene dynamics from temporal range sensing data
• Estimates optimal trajectories using Inverse Reinforcement Learning

Innovation

• Deep Learning-based Nonlinear Model Predictive Controller
• Inverse Reinforcement Learning for trajectory estimation
• Augmented Memory integrates sensing and states