Systematic Discovery of Semantic Attacks in Online Map Construction through Conditional Diffusion

2026-05-14

AI Summary

This work addresses the vulnerability of existing adversarial defenses to semantic-level attacks, particularly in the context of online high-definition map construction for autonomous driving, where realistic environmental variations—such as shadows or wet road surfaces—can compromise the recognition of critical road elements. The authors propose MIRAGE, a novel framework that leverages conditional diffusion models to generate semantically plausible yet misleading scene variants on the authentic data manifold, preserving topological consistency while enabling stealthy boundary deletion and fictitious boundary injection. Evaluated on the nuScenes dataset, MIRAGE evades mainstream defense mechanisms, causing 57.7% of lane boundary detections to fail and degrading 96% of motion planning trajectories. Critically, 80–84% of the generated scenes are judged as realistic by both human observers and vision-language models, substantially outperforming AdvPatch (0–9%) and exposing fundamental weaknesses in current defenses against semantic perturbations.

Abstract

Autonomous vehicles depend on online HD map construction to perceive lane boundaries, dividers, and pedestrian crossings -- safety-critical road elements that directly govern motion planning. While existing pixel perturbation attacks can disrupt the mapping, they can be neutralized by standard adversarial defenses. We present MIRAGE, a framework for systematic discovery of semantic attacks that bypass adversarial defenses and degrade mapping predictions by finding plausible environmental variation (e.g. shadows, wet roads). MIRAGE exploits the latent manifold of real-world data learned by diffusion models, and searches for semantically mutated scenes neighboring the ground truth with the same road topology yet mislead the mapping predictions. We evaluate MIRAGE on nuScenes and demonstrate two attacks: (1) boundary removal, suppressing 57.7% of detections and corrupting 96% of planned trajectories; and (2) boundary injection, the only method that successfully injects fictitious boundaries, while pixel PGD and AdvPatch fail entirely. Both attacks remain potent under various adversarial defenses. We use two independent VLM judges to quantify realism, where MIRAGE passes as realistic 80--84% of the time (vs. 97--99% for clean nuScenes), while AdvPatch only 0--9%. Our findings expose a categorical gap in current adversarial defenses: semantic-level perturbations that manifest as legitimate environmental variation are substantially harder to mitigate than pixel-level perturbations.

Problem

• semantic attacks
• online HD map construction
• adversarial defenses
• environmental variation
• autonomous driving

Innovation

• semantic attacks
• conditional diffusion
• HD map construction
• adversarial robustness
• environmental variation