Empirical assessment of the perception of graphical threat model acceptability

2025-12-03

AI Summary

Prior threat modeling evaluations have largely overlooked human factors, particularly for non-technical users, leaving a gap in understanding the acceptability of graphical threat models. Method: This study conducts the first systematic, user-perception–driven comparison of three prominent graphical threat modeling formalisms—Attack-Defense Trees (ADTs), Attack Graphs (AGs), and CORAS—using a controlled lab experiment. It employs a Latin-square task design, qualitative analysis, and a structured questionnaire grounded in the Model of Effort and Meaning (MEM) to assess usability and applicability across multiple dimensions. Contribution/Results: ADTs and CORAS significantly outperform AGs in comprehensibility, expressiveness, and practical utility. AGs suffer notably from perceived low practicality due to the absence of dedicated modeling tools. This work fills a critical gap in human-centered threat modeling evaluation and provides empirically grounded guidance for selecting lightweight, accessible security modeling approaches tailored to non-technical stakeholders.

Technology Category

• Humans and AI: User Experience and Usability
• Reasoning under Uncertainty: Graphical Models
• Knowledge Representation and Reasoning: Argumentation

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Threat modeling (TM) is an important aspect of risk analysis and secure software engineering. Graphical threat models are a recommended tool to analyze and communicate threat information. However, the comparison of different graphical threat models, and the acceptability of these threat models for an audience with a limited technical background, is not well understood, despite these users making up a sizable portion of the cybersecurity industry. We seek to compare the acceptability of three general, graphical threat models, Attack-Defense Trees (ADTs), Attack Graphs (AGs), and CORAS, for users with a limited technical background. We conducted a laboratory study with 38 bachelor students who completed tasks with the three threat models across three different scenarios assigned using a Latin square design. Threat model submissions were qualitatively analyzed, and participants filled out a perception questionnaire based on the Method Evaluation Model (MEM). We find that both ADTs and CORAS are broadly acceptable for a wide range of scenarios, and both could be applied successfully by users with a limited technical background; further, we also find that the lack of a specific tool for AGs may have impacted the perceived usefulness of AGs. We can recommend that users with a limited technical background use ADTs or CORAS as a general graphical TM method. Further research on the acceptability of AGs to such an audience and the effect of a dedicated TM tool support is needed.

Problem

• Evaluates acceptability of graphical threat models for non-technical users
• Compares Attack-Defense Trees, Attack Graphs, and CORAS models
• Assesses impact of tool support on perceived usefulness of models

Innovation

• Comparing graphical threat models for non-technical users
• Evaluating acceptability through laboratory study and questionnaires
• Recommending ADTs and CORAS for limited technical background users