🤖 AI Summary
Scalability limitations in reachability analysis and strong distributional assumptions in statistical verification hinder safety verification of high-dimensional autonomous systems. Method: This paper proposes a distributionally robust statistical verification framework for black-box systems. It introduces imprecise probability modeling to represent uncertainty, implements an imprecise neural network ensemble architecture, integrates active learning with the formal verification tool Sherlock for cooperative adaptive sampling, and employs distributionally robust optimization to provide provable safety guarantees over broad classes of distributions. Contribution/Results: Evaluated on multi-task OpenAI Gym MuJoCo benchmarks, the framework significantly improves scalability and confidence in safety verification, overcoming both the dimensionality curse and restrictive distributional assumptions inherent in conventional approaches.
📝 Abstract
A particularly challenging problem in AI safety is providing guarantees on the behavior of high-dimensional autonomous systems. Verification approaches centered around reachability analysis fail to scale, and purely statistical approaches are constrained by the distributional assumptions about the sampling process. Instead, we pose a distributionally robust version of the statistical verification problem for black-box systems, where our performance guarantees hold over a large family of distributions. This paper proposes a novel approach based on uncertainty quantification using concepts from imprecise probabilities. A central piece of our approach is an ensemble technique called Imprecise Neural Networks, which provides the uncertainty quantification. Additionally, we solve the allied problem of exploring the input set using active learning. The active learning uses an exhaustive neural-network verification tool Sherlock to collect samples. An evaluation on multiple physical simulators in the openAI gym Mujoco environments with reinforcement-learned controllers demonstrates that our approach can provide useful and scalable guarantees for high-dimensional systems.
