Neural optimal controller for stochastic systems via pathwise HJB operator

2024-02-23

arXiv.org

AI Summary

High-dimensional stochastic optimal control remains challenging due to the curse of dimensionality and limitations of conventional approaches relying on probabilistic representations of the Hamilton–Jacobi–Bellman (HJB) equation. Method: This paper proposes a physics-informed deep learning framework grounded in a pathwise HJB operator, unifying modeling and solution. It introduces the pathwise HJB operator as a novel physical constraint and designs two tailored numerical schemes—accommodating both explicit and implicit optimal control structures—integrated with PINNs, dynamic programming, SDE discretization, and HJB path-integral representations. Contribution/Results: A unified theoretical analysis quantifies truncation, approximation, and optimization errors. The framework significantly improves control accuracy and generalization across diverse high-dimensional tasks while ensuring interpretability and analytical tractability, establishing a new paradigm for real-time optimal control of complex stochastic systems.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The aim of this work is to develop deep learning-based algorithms for high-dimensional stochastic control problems based on physics-informed learning and dynamic programming. Unlike classical deep learning-based methods relying on a probabilistic representation of the solution to the Hamilton--Jacobi--Bellman (HJB) equation, we introduce a pathwise operator associated with the HJB equation so that we can define a problem of physics-informed learning. According to whether the optimal control has an explicit representation, two numerical methods are proposed to solve the physics-informed learning problem. We provide an error analysis on how the truncation, approximation and optimization errors affect the accuracy of these methods. Numerical results on various applications are presented to illustrate the performance of the proposed algorithms.

Problem

• Solving high-dimensional stochastic control problems
• Using physics-informed neural networks for HJB equations
• Developing efficient deep learning control methods

Innovation

• Physics-informed neural networks solve control problems
• Decoupled neural network parameterizes control and value
• HJB equation enforced via penalty terms in loss