Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective

2026-07-06

AI Summary

This study addresses the critical yet underexplored challenge of environmental illusions—such as shadows, reflections, and tire marks—that severely degrade lane perception in autonomous driving systems. To enable systematic evaluation, the authors introduce LanEvil++, the first high-fidelity, controllable benchmark for lane perception robustness, encompassing 14 illusion categories across 94 diverse 3D scenes. They further propose MIDA, a multimodal illusion defense approach to enhance model resilience. Experimental results demonstrate that environmental illusions reduce the accuracy and F1-score of state-of-the-art lane detection models by 5.27% and 10.49% on average, respectively. In contrast, MIDA improves robustness by 4.23% on conventional models and by 3.82% on vision-language models, significantly mitigating safety risks induced by perceptual illusions.

Abstract

Environmental illusions (eg., shadows, reflections, and tire marks) are naturally existing yet overlooked phenomena in real-world driving environments. They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving (AD) systems. However, existing researches largely overlook these phenomena, leaving a critical gap. To address this issue, we study AD robustness through the lane perception perspective, a fundamental task supporting core functions like cruise control and lane centering. We focus on two representative models: conventional lane detection (LD) and vision-language model-based systems (ADVLMs). In this work, we introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions. LanEvil++ encompasses 14 types of illusions and leverages the CARLA simulator to generate 94 high-fidelity, fully controllable 3D scenes, yielding a dataset of 90,292 annotated images, 1,596 video clips, and 41,855 visual question answering pairs. Extensive evaluations demonstrate that environmental illusions substantially degrade the performance of state-of-the-art LD methods. On average, LD models experience a 5.27% drop in Accuracy and a 10.49% decline in F1-score, while ADVLMs show a 2.03% reduction in GPT-score and a 0.75% drop in Language-score. Among all illusions, shadows emerge as the most disruptive factor, reducing accuracy by up to 7.20%. Furthermore, closed-loop simulations reveal that these illusions can lead to incorrect driving decisions. Complementary real-world case studies highlight safety-critical failures in actual traffic scenes. To enhance robustness, we propose the Multimodal Illusion Defense Approach (MIDA). MIDA achieves substantial gains under challenging conditions, boosting robustness by 4.23% on LD models and 3.82% on ADVLMs.

Problem

• environmental illusions
• autonomous driving
• lane perception
• robustness
• visual perception

Innovation

• environmental illusions
• lane perception
• robustness benchmark
• multimodal defense
• autonomous driving