🤖 AI Summary
To address the challenge of simultaneously ensuring safety and optimizing performance in uncertain dynamical systems, this paper proposes a safety-critical learning framework based on generalized action governors (AGs). The framework unifies the modeling of diverse safety enforcement mechanisms for multiple system classes and rigorously integrates AGs with both reinforcement learning (RL) and Koopman operator-based control, guaranteeing strict satisfaction of state constraints throughout the entire learning process. Methodologically, it encompasses AG synthesis, constraint analysis for linear and discrete-time systems, safety-aware RL, data-driven Koopman model identification, and real-time feasibility verification. We provide theoretical guarantees on closed-loop stability and safety of the AG-augmented system. Numerical experiments demonstrate that the two proposed safe learning algorithms achieve significant improvements in closed-loop performance—without any constraint violations.
📝 Abstract
This article introduces a general framework for safe control and learning based on the generalized action governor (AG). The AG is a supervisory scheme for augmenting a nominal closed-loop system with the ability of strictly handling prescribed safety constraints. In the first part of this article, we present a generalized AG methodology and analyze its key properties in a general setting. Then, we introduce tailored AG design approaches derived from the generalized methodology for linear and discrete systems. Afterward, we discuss the application of the generalized AG to facilitate safe online learning, which aims at safely evolving control parameters using real-time data to enhance control performance in uncertain systems. We present two safe learning algorithms based on, respectively, reinforcement learning and data-driven Koopman operator-based control integrated with the generalized AG to exemplify this application. Finally, we illustrate the developments with a numerical example.