Infinite-Horizon Value Function Approximation for Model Predictive Control

2025-02-10

AI Summary

To address the challenge of simultaneously ensuring real-time performance, safety, global closed-loop stability, and hard constraint satisfaction in Model Predictive Control (MPC), this paper proposes a deep neural network-based approximation of the infinite-horizon value function. The network is trained via joint value iteration and nonlinear trajectory optimization to learn a constraint-admissible optimal terminal cost. This work presents the first provably globally stable neural parameterization of an infinite-horizon value function for MPC; when embedded online, it eliminates the need for manual terminal cost design while inherently guaranteeing closed-loop stability. Evaluated on standard simulation benchmarks and a real-world industrial robotic arm performing online obstacle avoidance, the method significantly improves constraint satisfaction rates and closed-loop stability, while enabling complex real-time tasks such as dynamic obstacle evasion.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Model Predictive Control has emerged as a popular tool for robots to generate complex motions. However, the real-time requirement has limited the use of hard constraints and large preview horizons, which are necessary to ensure safety and stability. In practice, practitioners have to carefully design cost functions that can imitate an infinite horizon formulation, which is tedious and often results in local minima. In this work, we study how to approximate the infinite horizon value function of constrained optimal control problems with neural networks using value iteration and trajectory optimization. Furthermore, we demonstrate how using this value function approximation as a terminal cost provides global stability to the model predictive controller. The approach is validated on two toy problems and a real-world scenario with online obstacle avoidance on an industrial manipulator where the value function is conditioned to the goal and obstacle.

Problem

• Approximate infinite horizon value function
• Ensure global stability in MPC
• Apply neural networks in control problems

Innovation

• Neural networks approximate value functions
• Value iteration ensures global stability
• Terminal cost enhances model predictive control