Discriminative Barrier Functions for Safe Adversarial Imitation Learning from Observation

2026-07-15

AI Summary

This work addresses the safety risks inherent in inverse reinforcement learning (IRL) under unconstrained exploration and the difficulty of manually designing control barrier functions (CBFs) that generalize from observed data. The authors propose a novel framework that constrains the IRL reward function to adhere to the structure of a CBF, enabling, for the first time, the data-driven recovery of safety-critical barrier functions directly from unlabeled expert trajectories. By integrating adversarial imitation learning with CBF theory, the method ensures robustness against unseen hazardous states while maintaining online safety during policy optimization. Experimental results demonstrate significant improvements over standard IRL baselines in simulated navigation tasks and validate the approach’s effectiveness in real-world obstacle avoidance scenarios.

Abstract

Inverse Reinforcement Learning (IRL) algorithms are powerful tools for learning from and generalizing expert demonstrations, but they often rely on unconstrained exploration, rendering them unsafe for real-world deployment. Meanwhile, Control Barrier Functions (CBFs) can guarantee the safety of control systems, but the analytical design of CBFs can be time-consuming and esoteric. In this work, we address these limitations jointly by constraining reward function candidacy during IRL to the space of CBFs, yielding a formulation that exhibits safe online control with continuous experiential improvement. Crucially, this framework enables the data-driven recovery of barrier functions directly from unlabeled expert observations. We demonstrate that the recovered barrier function is robust to unsafe states entirely absent from the expert data. Furthermore, we benchmark our method against standard IRL baselines in a simulated navigation environment, demonstrating improved safety performance. Finally, we investigate the trade-offs of planning-based versus policy-based IRL methods across both simulation and a real world obstacle avoidance task.

Problem

• Inverse Reinforcement Learning
• Control Barrier Functions
• Safe Imitation Learning
• Adversarial Learning
• Safety Guarantees

Innovation

• Control Barrier Functions
• Inverse Reinforcement Learning
• Safe Imitation Learning
• Data-driven Safety
• Unlabeled Expert Observations