Problem
Research questions and friction points this paper is trying to address.
operational-data fidelity
software reliability assessment
safety-critical systems
autonomous vehicles
failure data granularity
The Impact of Operational-Data Fidelity when Assessing Safety-Critical Autonomous-Vehicle Software
🤖 AI Summary
Low-granularity operational data can lead to overly optimistic assessments of autonomous driving software reliability, thereby undermining the credibility of safety certification. This work proposes a systematic approach based on Conservative Bayesian Inference (CBI) to quantify, for the first time, the adverse impact of insufficient data fidelity on the robustness of reliability claims. By integrating statistical robustness analysis with software reliability modeling, the study demonstrates that even conservative inference strategies may yield misleading conclusions when applied to low-fidelity data. The paper establishes the first conservative estimation framework that explicitly accounts for the influence of data granularity on reliability assessment, highlighting the critical importance of high-fidelity operational data in safety certification of autonomous driving systems.
📝 Abstract
For safety-critical software, data from the software's operational past (e.g. a sequence of success and failure events experienced by the software) can provide strong statistical support for reliability claims about the software. However, such data might not describe past software failure events in sufficient detail, and this might leave a reliability assessment (based on this data) unable to account for important features of past software failures. In this paper, by extending conservative Bayesian inference (CBI) techniques used in reliability assessment, we illustrate a principled statistical approach for checking the robustness of reliability claims derived from insufficiently detailed operational data. We demonstrate the extent to which insufficient detail in operational data can undermine software reliability claims in autonomous vehicle (AV) safety assessment scenarios. Reliability claims derived from insufficiently fine-grained data might be dangerously optimistic, despite a concerted effort by an assessor to use such data conservatively during the assessment. While these findings are consistent with previous work on the impact of statistical model fidelity in Bayesian software reliability assessments, our work clarifies why attempts to use low-fidelity data conservatively can be naive, and we give the first conservative estimates of the impact of data fidelity on assessments.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Conservative Bayesian Inference
Data Fidelity
Software Reliability Assessment
Autonomous Vehicles
Operational Data
🔎 Similar Papers
No similar papers found.
