Testing the Fault-Tolerance of Multi-Sensor Fusion Perception in Autonomous Driving Systems

2025-04-18

AI Summary

Current multi-sensor fusion (MSF) perception systems for high-level autonomous driving lack robust fault tolerance under camera and LiDAR failures. Method: This work introduces the first realistic sensor-failure model, integrated into the Baidu Apollo platform; proposes FADE—a feedback-guided differential fuzzing framework specifically designed for evaluating MSF fault tolerance—and conducts closed-loop validation via simulation and real-vehicle testing (Apollo 6.0 EDU). Contribution/Results: FADE efficiently uncovers diverse safety-violating scenarios under sensor faults. Experiments demonstrate that representative failures significantly degrade decision-making reliability, exposing critical vulnerabilities in MSF architectures. To our knowledge, this is the first end-to-end evaluation pipeline spanning realistic fault modeling, differential fuzzing, and physical-world validation—establishing a novel, reproducible paradigm for assessing perception fault tolerance in autonomous driving systems.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Adversarial Attacks & Robustness
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

High-level Autonomous Driving Systems (ADSs), such as Google Waymo and Baidu Apollo, typically rely on multi-sensor fusion (MSF) based approaches to perceive their surroundings. This strategy increases perception robustness by combining the respective strengths of the camera and LiDAR and directly affects the safety-critical driving decisions of autonomous vehicles (AVs). However, in real-world autonomous driving scenarios, cameras and LiDAR are subject to various faults, which can probably significantly impact the decision-making and behaviors of ADSs. Existing MSF testing approaches only discovered corner cases that the MSF-based perception cannot accurately detected by MSF-based perception, while lacking research on how sensor faults affect the system-level behaviors of ADSs. To address this gap, we conduct the first exploration of the fault tolerance of MSF perception-based ADS for sensor faults. In this paper, we systematically and comprehensively build fault models for cameras and LiDAR in AVs and inject them into the MSF perception-based ADS to test its behaviors in test scenarios. To effectively and efficiently explore the parameter spaces of sensor fault models, we design a feedback-guided differential fuzzer to discover the safety violations of MSF perception-based ADS caused by the injected sensor faults. We evaluate FADE on the representative and practical industrial ADS, Baidu Apollo. Our evaluation results demonstrate the effectiveness and efficiency of FADE, and we conclude some useful findings from the experimental results. To validate the findings in the physical world, we use a real Baidu Apollo 6.0 EDU autonomous vehicle to conduct the physical experiments, and the results show the practical significance of our findings.

Problem

• Testing fault-tolerance in multi-sensor fusion for autonomous driving
• Exploring sensor faults' impact on autonomous vehicle decision-making
• Developing feedback-guided fuzzer to detect safety violations

Innovation

• Build fault models for cameras and LiDAR
• Use feedback-guided differential fuzzer
• Validate findings with physical experiments