A Taxonomy of Real-World Defeaters in Safety Assurance Cases

2025-02-01

AI Summary

Safety-critical cyber-physical systems (CPS) rely on assurance cases to substantiate safety claims, yet existing approaches lack systematic, empirically grounded methods for identifying and classifying defeaters—counters to the validity or sufficiency of assurance arguments. Method: Drawing on a 20-year systematic literature review and open coding of real-world assurance cases, this work proposes the first empirically derived, seven-dimensional defeater taxonomy. Contribution/Results: The taxonomy categorizes defeaters along seven dimensions—including insufficient evidence, broken inference chains, and ambiguous claims—thereby establishing the first standardized framework for defect analysis in assurance cases. We release an open-source, reusable defeater classification ontology and supporting toolkit, enabling consistent interpretation, reproducible evaluation, and scalable management of defeaters. This provides the first benchmark framework explicitly designed to enhance the credibility and trustworthiness of safety arguments for both industry and academia.

Technology Category

• Knowledge Representation and Reasoning: Argumentation
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The rise of cyber-physical systems in safety-critical domains calls for robust risk-evaluation frameworks. Assurance cases, often required by regulatory bodies, are a structured approach to demonstrate that a system meets its safety requirements. However, assurance cases are fraught with challenges, such as incomplete evidence and gaps in reasoning, called defeaters, that can call into question the credibility and robustness of assurance cases. Identifying these defeaters increases confidence in the assurance case and can prevent catastrophic failures. The search for defeaters in an assurance case, however, is not structured, and there is a need to standardize defeater analysis. The software engineering community thus could benefit from having a reusable classification of real-world defeaters in software assurance cases. In this paper, we conducted a systematic study of literature from the past 20 years. Using open coding, we derived a taxonomy with seven broad categories, laying the groundwork for standardizing the analysis and management of defeaters in safety-critical systems. We provide our artifacts as open source for the community to use and build upon, thus establishing a common framework for understanding defeaters.

Problem

• Cybersecurity
• Argumentation
• Evidence Evaluation

Innovation

• Unified Framework
• Cybersecurity Rebuttal Points
• Standardization Method