A Scalability Analysis of Quantitative Confidence Assessment Methods for Assurance Cases

2026-06-13

AI Summary

This study addresses the limited applicability of quantitative confidence assessment methods in assurance cases due to high decision complexity and human effort. It presents the first systematic modeling and comparison of the scalability of Bayesian Belief Networks, Dempster–Shafer theory, and Certus under worst-case and average-case scenarios, leveraging parameters from published case studies to construct workload and complexity estimation models. The findings reveal that although Certus exhibits the highest worst-case computational complexity, it incurs significantly lower average-case effort than the other approaches, demonstrating superior practical scalability. This provides developers with a reliable tool for predicting assessment costs in real-world assurance contexts.

Abstract

This paper proposes a model to estimate the decision complexity and effort required to apply quantitative confidence assessment methods to assurance cases. The model considers both the worst and average case for these measures and characterizes how these quantities scale with argument size. Prior work has indicated that the additional effort required to apply these methods is a barrier to their adoption by assurance case practitioners. Researchers developing new methods, or improving existing methods, can use this model to estimate the effort required to apply their method. The proposed model is parameterized using data from published case studies and is applied to three existing quantitative confidence assessment methods: the Bayesian Belief Network method, the Dempster-Shafer Theory method, and the Certus method. The results show that, while Certus has the highest worst-case decision complexity, its average-case effort is lower than the BBN and DST methods.

Problem

• scalability
• quantitative confidence assessment
• assurance cases
• decision complexity
• effort estimation

Innovation

• scalability analysis
• quantitative confidence assessment
• assurance cases
• decision complexity
• effort estimation