A Physics-Informed Learning Framework to Solve the Infinite-Horizon Optimal Control Problem

2025-05-28

AI Summary

Physics-informed neural networks (PINNs) struggle to converge to the optimal value function for infinite-horizon optimal control of nonlinear systems due to the non-uniqueness of solutions to the steady-state Hamilton–Jacobi–Bellman (HJB) equation. Method: This paper proposes a PINN framework based on a finite-horizon HJB variant. It replaces the ill-posed steady-state HJB with a uniquely solvable finite-horizon counterpart and introduces a verifiable, time-horizon adaptive extension mechanism to ensure consistent approximation of the optimal value function—without requiring a pre-designed stabilizing controller or iterative policy evaluation. Contribution/Results: We establish theoretical guarantees of uniform convergence of the approximated value function to the true optimal one. Numerical simulations demonstrate high-accuracy, robust closed-loop control performance. The method accommodates non-polynomial basis functions, significantly enhancing generalization capability and computational stability for complex nonlinear systems.

Technology Category

• Search and Optimization: Adversarial Search
• Multiagent Systems: Adversarial Agents
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We propose a physics-informed neural networks (PINNs) framework to solve the infinite-horizon optimal control problem of nonlinear systems. In particular, since PINNs are generally able to solve a class of partial differential equations (PDEs), they can be employed to learn the value function of the infinite-horizon optimal control problem via solving the associated steady-state Hamilton-Jacobi-Bellman (HJB) equation. However, an issue here is that the steady-state HJB equation generally yields multiple solutions; hence if PINNs are directly employed to it, they may end up approximating a solution that is different from the optimal value function of the problem. We tackle this by instead applying PINNs to a finite-horizon variant of the steady-state HJB that has a unique solution, and which uniformly approximates the optimal value function as the horizon increases. An algorithm to verify if the chosen horizon is large enough is also given, as well as a method to extend it -- with reduced computations and robustness to approximation errors -- in case it is not. Unlike many existing methods, the proposed technique works well with non-polynomial basis functions, does not require prior knowledge of a stabilizing controller, and does not perform iterative policy evaluations. Simulations are performed, which verify and clarify theoretical findings.

Problem

• Solving infinite-horizon optimal control for nonlinear systems
• Addressing multiple solutions in steady-state HJB equations
• Verifying and extending horizon size for optimal approximation

Innovation

• Physics-informed neural networks solve control problems
• Finite-horizon HJB ensures unique optimal solution
• Algorithm verifies and extends horizon efficiently