WILD SAM: A Simulated-and-Real Data Augmentation for Autonomous Driving Perception under Challenging Weather

2026-05-01

AI Summary

This work addresses the significant performance degradation of autonomous driving perception models under adverse weather conditions caused by domain shift. To mitigate this issue, the authors propose the WILD framework, which introduces a pseudo-label denoising mechanism into weather domain adaptation for the first time. By integrating cross-domain self-training with simulation-to-real hybrid data augmentation, WILD enables robust fine-tuning of object detection models. The approach effectively alleviates performance deterioration in challenging conditions such as rain and snow, achieving up to a 13% improvement in average precision (AP) on the Four Seasons dataset and substantially narrowing the performance gap compared to clear-weather scenarios.

Abstract

The performance of state-of-the-art object detectors degrades significantly under adverse weather, causing a safety-critical domain shift problem for autonomous vehicles. Recent efforts address this problem by relying on synthetic data to train the object detectors, which limits their real-world applicability. Meanwhile, pseudo-labeling is widely used for cross-dataset domain adaptation problems. However, these methods have not been exploited by weather-based domain adaptation approaches due to the noisy nature of such labels generated under harsh weather conditions. In this paper, we propose two new approaches to mitigate this weather-induced domain shift. First, we propose a Weather-Induced pseudo Label Denoising (WILD) framework that filters noisy pseudo labels generated by real data captured under adverse weather conditions. Second, we develop a novel hybrid training methodology, WILD SAM, that exploits both pseudo-label denoising and simulation-based training solutions while using real-data from the target harsh-weather domain. We validate both proposed approaches, WILD and WILD SAM, on the recently released Four Seasons dataset across rainy and snowy scenarios. Experiments show that the proposed frameworks improve Average Precision (AP) up to 13\% and significantly reduce the weather-induced performance gap relative to the baseline. The code is available at: https://github.com/Kh-Hamed/WILD-SAM

Problem

• domain shift
• adverse weather
• autonomous driving
• object detection
• pseudo-labeling

Innovation

• pseudo-label denoising
• domain adaptation
• autonomous driving
• adverse weather
• data augmentation