Data-Driven Falsification of Cyber-Physical Systems

2024-02-22

International Symposium on Electronic Commerce

AI Summary

This paper addresses the challenge of detecting unsafe behaviors in safety-critical cyber-physical systems (CPS). We propose NNFal, a data-driven falsification framework designed to efficiently discover counterexamples—rather than prove their absence. Methodologically, NNFal first learns a deep neural network (DNN) surrogate model of CPS dynamics from simulation data, then adapts adversarial attack techniques from DNN robustness verification (e.g., PGD, Semi-Gradient) to generate counterexamples in a simulation-driven, black-box or white-box manner. Our key contribution is the first systematic mapping between CPS safety falsification and DNN robustness falsification, enabling effective discovery of hard-to-find counterexamples for both linear and nonlinear dynamical systems. Extensive evaluation on multiple CPS benchmarks demonstrates that NNFal significantly outperforms state-of-the-art approaches in falsification efficiency, effectiveness, and cross-model generalizability.

Technology Category

• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements

Abstract

Cyber-Physical Systems (CPS) are abundant in safety-critical domains such as healthcare, avionics, and autonomous vehicles. The formal verification of their operational safety is therefore of utmost importance. In this paper, we address the falsification problem where the focus is on searching for an unsafe execution in the system instead of proving their absence. The contribution of this paper is a framework that connects the falsification of the safety properties of CPS with the falsification of deep neural networks (DNNs). This connection is established by: (1) Constructing a DNN model of the CPS under test and (2) The application of various falsification tools of DNNs to falsify CPS. The proposed framework has the potential to exploit a repertoire of adversarial attack algorithms designed for the falsification of robustness properties of DNNs. Although the proposed technique is applicable to systems in general that can be executed/simulated, we demonstrate its effectiveness, particularly in CPS. We show that our framework implemented as a prototypical tool NNFal can detect hard-to-find counterexamples in CPS having linear as well as non-linear dynamics.

Problem

• Falsifying Cyber-Physical Systems by finding unsafe executions
• Leveraging Decision Trees for faster CPS falsification
• Applying DNN adversarial attacks to CPS falsification

Innovation

• Surrogate model using DNN or Decision Tree
• DNN falsification tools for CPS
• Decision Tree-guided novel falsification algorithm