Towards Safety-Aware Mutation Testing for Autonomous Driving Systems

2026-06-24

AI Summary

This work addresses the lack of systematic criteria for determining safe termination timing in autonomous driving simulation testing and the inability of conventional coverage metrics to capture critical failures arising from inter-module interactions. To overcome these limitations, the paper proposes Safety-Aware Mutation Testing (SAMT), which integrates system safety analysis methods—such as System-Theoretic Process Analysis (STPA)—into mutation testing to generate semantically meaningful mutants that reflect real-world hazards at the module interaction level. By leveraging STPA-derived safety rules to guide mutation generation, injecting faults at the message level, and modeling temporal constraints, SAMT enables automated scenario construction and rigorous assessment of test adequacy. This approach provides a theoretical foundation for informed test termination decisions and targeted system remediation.

Abstract

Simulation-based testing is essential for ensuring the safety of Autonomous Driving Systems (ADS), yet the community lacks a systematic criterion for determining when we can safely stop additional test scenario generation. Existing coverage metrics typically focus on individual component reliability or treat the ADS as a black box, failing to capture certain component interactions that cause most ADS accidents. While traditional mutation testing provides a falsifiable measure of test adequacy, directly porting code- and deep learning model-level mutations to the corresponding modules of ADS is insufficient. In this vision paper, we propose a paradigm shift toward Safety-Aware Mutation Testing (SAMT). Unlike traditional mutation testing, which creates mutants (i.e., faulty versions of the software under test) by injecting artificial faults into individual components, SAMT systematically injects temporally bounded faults into the messages exchanged between ADS modules to simulate realistic interaction failures. To ensure these mutants represent genuine hazards, we propose deriving mutant generation rules directly from top-down safety engineering frameworks, such as System-Theoretic Process Analysis (STPA). By embedding systems thinking into the mutation testing pipeline, SAMT provides a rigorous mechanism for evaluating test adequacy, enabling automated scenario generation, and guiding ADS repair. We also outline critical open challenges.

Problem

• Autonomous Driving Systems
• Mutation Testing
• Safety Assurance
• Test Adequacy
• Component Interaction

Innovation

• Safety-Aware Mutation Testing
• Autonomous Driving Systems
• System-Theoretic Process Analysis
• Interaction Faults
• Test Adequacy