Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learning

2026-04-14

AI Summary

Existing control barrier function (CBF) approaches rely on explicit structural knowledge of system dynamics and uncertainty models, limiting their applicability to general nonlinear systems and often yielding overly conservative safe sets. This work proposes a model-free robust Q-CBF framework that, for the first time, integrates the safety value function with the Q-function to formulate robust safety constraints directly in the state-action space. By leveraging adversarial reinforcement learning and solving the associated Hamilton–Jacobi–Isaacs equation, the method computes the largest possible robustly safe set. Evaluations on an inverted pendulum and a 36-dimensional quadrupedal robot demonstrate that the proposed approach significantly reduces conservatism and achieves more reliable safe control under unknown disturbances.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Robust control barrier functions (CBFs) provide a principled mechanism for smooth safety enforcement under worst-case disturbances. However, existing approaches typically rely on explicit, closed-form structure in the dynamics (e.g., control-affine) and uncertainty models. This has led to limited scalability and generality, with most robust CBFs certifying only conservative subsets of the maximal robust safe set. In this paper, we introduce a new robust CBF framework for general nonlinear systems under bounded uncertainty. We first show that the safety value function solving the dynamic programming Isaacs equation is a valid robust discrete-time CBF that enforces safety on the maximal robust safe set. We then adopt the key reinforcement learning (RL) notion of quality function (or Q-function), which removes the need for explicit dynamics by lifting the barrier certificate into state-action space and yields a novel robust Q-CBF constraint for safety filtering. Combined with adversarial RL, this enables the synthesis and deployment of robust Q-CBFs on general nonlinear systems with black-box dynamics and unknown uncertainty structure. We validate the framework on a canonical inverted pendulum benchmark and a 36-D quadruped simulator, achieving substantially less conservative safe sets than barrier-based baselines on the pendulum and reliable safety enforcement even under adversarial uncertainty realizations on the quadruped.

Problem

• robust control barrier functions
• nonlinear systems
• bounded uncertainty
• maximal robust safe set
• black-box dynamics

Innovation

• Robust Control Barrier Functions
• Adversarial Reinforcement Learning
• Q-CBF
• Maximal Robust Safe Set
• Black-box Dynamics