Lyapunov Neural ODE State-Feedback Control Policies

2024-08-31

AI Summary

This work addresses the optimal stabilization problem for continuous-time nonlinear systems subject to state and control constraints. We propose a state-feedback control method based on Neural Ordinary Differential Equations (NODEs). Our key contribution is a novel Lyapunov-based loss mechanism that directly incorporates an exponentially stabilizing Control Lyapunov Function (CLF) into the training objective—thereby guaranteeing, for the first time within the NODE framework, exponential stability of the closed-loop system and robustness against initial-state perturbations. The feedback policy is parameterized by a deep neural network, and stability constraints are enforced via constrained optimization during training. Evaluated on tasks such as plasma medicine dose delivery, the method achieves rapid, robust stabilization while satisfying hard state and control constraints. It significantly reduces both convergence time and inference latency compared to baseline approaches.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Deep neural networks are increasingly used as an effective way to represent control policies in various learning-based control paradigms. For continuous-time optimal control problems (OCPs), which are central to many decision-making tasks, control policy learning can be cast as a neural ordinary differential equation (NODE) problem wherein state and control constraints are naturally accommodated. This paper presents a NODE approach to solving continuous-time OCPs for the case of stabilizing a known constrained nonlinear system around an equilibrium state. The approach, termed Lyapunov-NODE control (L-NODEC), uses a novel Lyapunov loss formulation that incorporates an exponentially-stabilizing control Lyapunov function to learn a state-feedback neural control policy. The proposed Lyapunov loss allows L-NODEC to guarantee exponential stability of the controlled system, as well as its adversarial robustness to perturbations to the initial state. The performance of L-NODEC is illustrated in two problems, including a dose delivery problem in plasma medicine, wherein L-NODEC effectively stabilizes the controlled system around the equilibrium state despite perturbations to the initial state and reduces the inference time necessary to reach equilibrium.

Problem

• Solving continuous-time optimal control problems
• Stabilizing nonlinear systems around equilibrium
• Ensuring adversarial robustness to initial state perturbations

Innovation

• Neural ODE for control
• Lyapunov loss for stability
• Adversarial robustness in control