Learning Safe Control via On-the-Fly Bandit Exploration

2025-06-12

AI Summary

To address safety filter failure—i.e., infeasibility of admissible control inputs—under high model uncertainty, this paper proposes an online safe learning framework that operates without fallback controllers. When safety constraints become infeasible, the method leverages Control Barrier Functions (CBFs) to guide Gaussian process–driven bounded active exploration, enabling closed-loop data collection that incrementally improves safety certification. It is the first approach to achieve safe learning under zero-mean prior dynamics without requiring backup controllers. By tightly integrating CBF-based safety verification with Bayesian active exploration, the framework theoretically guarantees simultaneous improvement in closed-loop safety and filter feasibility. Crucially, it eliminates reliance on conservative error bounds or auxiliary controllers, delivering rigorous, adaptive safety guarantees even under severe model uncertainty.

Technology Category

• Natural Language Processing: Safety and Robustness
• Reasoning under Uncertainty: Sequential Decision Making
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Control tasks with safety requirements under high levels of model uncertainty are increasingly common. Machine learning techniques are frequently used to address such tasks, typically by leveraging model error bounds to specify robust constraint-based safety filters. However, if the learned model uncertainty is very high, the corresponding filters are potentially invalid, meaning no control input satisfies the constraints imposed by the safety filter. While most works address this issue by assuming some form of safe backup controller, ours tackles it by collecting additional data on the fly using a Gaussian process bandit-type algorithm. We combine a control barrier function with a learned model to specify a robust certificate that ensures safety if feasible. Whenever infeasibility occurs, we leverage the control barrier function to guide exploration, ensuring the collected data contributes toward the closed-loop system safety. By combining a safety filter with exploration in this manner, our method provably achieves safety in a setting that allows for a zero-mean prior dynamics model, without requiring a backup controller. To the best of our knowledge, it is the first safe learning-based control method that achieves this.

Problem

• Ensuring safety in control tasks under high model uncertainty
• Addressing infeasibility of safety filters via on-the-fly exploration
• Achieving safe learning-based control without backup controllers

Innovation

• Uses Gaussian process bandit for on-the-fly exploration
• Combines control barrier function with learned model
• Ensures safety without requiring backup controller