V2X-DGW: Domain Generalization for Multi-agent Perception under Adverse Weather Conditions

2024-03-17

arXiv.org

AI Summary

To address the domain generalization challenge in LiDAR-based multi-agent V2X perception—where 3D detection performance drastically degrades under unseen adverse weather conditions (e.g., rain, fog, snow)—this paper proposes a novel training paradigm leveraging only clear-weather data to enhance cross-weather robustness. Our method introduces two core components: Adaptive Weather Augmentation (AWA) and a dual-alignment mechanism—Trusted-World Alignment (TWA) enforces weather-invariant feature representation, while Agent-wise Contrastive Alignment (ACA) promotes consistent perceptual embeddings across agents. Integrating physics-informed weather simulation, domain-generalization learning, and trusted-domain regularization, our approach requires no annotations under adverse weather. Evaluated on newly constructed OPV2V-w and V2XSet-w benchmarks, it achieves 12.6–23.8% absolute mAP gains in unseen fog/rain/snow scenarios, significantly outperforming state-of-the-art methods.

Technology Category

• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Current LiDAR-based Vehicle-to-Everything (V2X) multi-agent perception systems have shown the significant success on 3D object detection. While these models perform well in the trained clean weather, they struggle in unseen adverse weather conditions with the domain gap. In this paper, we propose a Domain Generalization based approach, named extit{V2X-DGW}, for LiDAR-based 3D object detection on multi-agent perception system under adverse weather conditions. Our research aims to not only maintain favorable multi-agent performance in the clean weather but also promote the performance in the unseen adverse weather conditions by learning only on the clean weather data. To realize the Domain Generalization, we first introduce the Adaptive Weather Augmentation (AWA) to mimic the unseen adverse weather conditions, and then propose two alignments for generalizable representation learning: Trust-region Weather-invariant Alignment (TWA) and Agent-aware Contrastive Alignment (ACA). To evaluate this research, we add Fog, Rain, Snow conditions on two publicized multi-agent datasets based on physics-based models, resulting in two new datasets: OPV2V-w and V2XSet-w. Extensive experiments demonstrate that our V2X-DGW achieved significant improvements in the unseen adverse weathers. The code is available at https://github.com/Baolu1998/V2X-DGW.

Problem

• Improves LiDAR-based 3D object detection in adverse weather.
• Enhances multi-agent perception under unseen weather conditions.
• Uses clean weather data to generalize to adverse conditions.

Innovation

• Adaptive Weather Augmentation mimics adverse conditions
• Trust-region Weather-invariant Alignment enhances generalization
• Agent-aware Contrastive Alignment improves multi-agent perception