Sequence-Preserving Dual-FoV Defense for Traffic Sign and Light Recognition in Autonomous Vehicles

2025-10-02

AI Summary

Traffic sign and traffic light recognition in autonomous driving is vulnerable to digital adversarial attacks and natural perturbations (e.g., rain, fog, glare, graffiti), leading to perceptual failures and safety hazards; existing defenses lack joint modeling of temporal continuity, multi-field-of-view perception, and cross-domain (digital/physical) robustness. This paper proposes a dual-field-of-view, sequence-preserving, three-tier unified defense framework that integrates temporal modeling with multi-perspective feature fusion, incorporating feature compression, defensive distillation, entropy-based anomaly detection, and temporal-aligned voting. Evaluated under four complex operational scenarios—highway, nighttime, rainy, and urban—the framework achieves 79.8% mAP, reduces attack success rate to 18.2%, and decreases high-risk misclassifications by 32%, significantly outperforming YOLOv8/v9 and BEVFormer. The approach demonstrates strong practical deployability.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Traffic light and sign recognition are key for Autonomous Vehicles (AVs) because perception mistakes directly influence navigation and safety. In addition to digital adversarial attacks, models are vulnerable to existing perturbations (glare, rain, dirt, or graffiti), which could lead to dangerous misclassifications. The current work lacks consideration of temporal continuity, multistatic field-of-view (FoV) sensing, and robustness to both digital and natural degradation. This study proposes a dual FoV, sequence-preserving robustness framework for traffic lights and signs in the USA based on a multi-source dataset built on aiMotive, Udacity, Waymo, and self-recorded videos from the region of Texas. Mid and long-term sequences of RGB images are temporally aligned for four operational design domains (ODDs): highway, night, rainy, and urban. Over a series of experiments on a real-life application of anomaly detection, this study outlines a unified three-layer defense stack framework that incorporates feature squeezing, defensive distillation, and entropy-based anomaly detection, as well as sequence-wise temporal voting for further enhancement. The evaluation measures included accuracy, attack success rate (ASR), risk-weighted misclassification severity, and confidence stability. Physical transferability was confirmed using probes for recapture. The results showed that the Unified Defense Stack achieved 79.8mAP and reduced the ASR to 18.2%, which is superior to YOLOv8, YOLOv9, and BEVFormer, while reducing the high-risk misclassification to 32%.

Problem

• Defends against digital and natural perturbations in traffic recognition
• Addresses lack of temporal continuity in autonomous vehicle perception
• Improves robustness across multiple operational domains and conditions

Innovation

• Dual FoV sequence-preserving robustness framework
• Unified three-layer defense stack with temporal voting
• Multi-source dataset for four operational design domains