Learn, Check, Test -- Security Testing Using Automata Learning and Model Checking

2025-09-26

AI Summary

Addressing the challenge of modeling and formally verifying security in black-box industrial cyber-physical systems (CPS)—such as those in extended supply chains or classified environments—this paper proposes an end-to-end security analysis framework integrating active automata learning with model checking. Our key contribution is the Context-aware Propositional Mapping (CPM) mechanism, which automatically transforms learned Mealy machines into Kripke-like structures, enabling nondeterministic modeling, semantic enrichment, and scalable instantiation of safety properties. The method requires no internal system knowledge, relying solely on input-output interactions to infer behavioral models and verify security properties of communication protocols. We validate its generality, effectiveness, and toolchain reusability across multiple protocols—including NFC and UDS—demonstrating significant improvements in automation and formal assurance for black-box CPS security testing.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Cyber-physical systems are part of industrial systems and critical infrastructure. Therefore, they should be examined in a comprehensive manner to verify their correctness and security. At the same time, the complexity of such systems demands such examinations to be systematic and, if possible, automated for efficiency and accuracy. A method that can be useful in this context is model checking. However, this requires a model that faithfully represents the behavior of the examined system. Obtaining such a model is not trivial, as many of these systems can be examined only in black box settings due to, e.g., long supply chains or secrecy. We therefore utilize active black box learning techniques to infer behavioral models in the form of Mealy machines of such systems and translate them into a form that can be evaluated using a model checker. To this end, we will investigate a cyber-physical systems as a black box using its external communication interface. We first annotate the model with propositions by mapping context information from the respective protocol to the model using Context-based Proposition Maps (CPMs). We gain annotated Mealy machines that resemble Kripke structures. We then formally define a template, to transfer the structures model checker-compatible format. We further define generic security properties based on basic security requirements. Due to the used CPMs, we can instantiate these properties with a meaningful context to check a specific protocol, which makes the approach flexible and scalable. The gained model can be easily altered to introduce non-deterministic behavior (like timeouts) or faults and examined if the properties still. Lastly, we demonstrate the versatility of the approach by providing case studies of different communication protocols (NFC and UDS), checked with the same tool chain and the same security properties.

Problem

• Inferring behavioral models from black-box cyber-physical systems
• Translating learned models into model-checker compatible formats
• Automatically verifying security properties across communication protocols

Innovation

• Active black box learning infers Mealy machine models
• Context-based Proposition Maps annotate models for verification
• Generic security properties instantiated for protocol-specific checking