Adverse Weather-Independent Framework Towards Autonomous Driving Perception through Temporal Correlation and Unfolded Regularization

2025-08-03

AI Summary

Autonomous driving perception degrades significantly under adverse weather conditions (e.g., fog, rain), and existing domain adaptation methods rely on clean-image references and target only single-weather domains, failing to generalize to real-world composite weather scenarios. To address this, we propose Advent—a reference-free, weather-agnostic universal perception framework. Advent introduces three core components: (1) local temporal consistency modeling to exploit inter-frame structural coherence; (2) global feature shuffling to decouple weather-specific patterns from semantic representations; and (3) a deep-unfolding regularizer that enforces robustness via iterative refinement. Crucially, Advent eliminates dependence on clean-reference images and enables robust semantic segmentation across arbitrary combinations of adverse weather conditions. Evaluated on a multi-weather mixed benchmark, Advent substantially outperforms state-of-the-art methods, demonstrating superior cross-weather generalization and practical applicability for real-world autonomous systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Ad search and search for Web retail
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Various adverse weather conditions such as fog and rain pose a significant challenge to autonomous driving (AD) perception tasks like semantic segmentation, object detection, etc. The common domain adaption strategy is to minimize the disparity between images captured in clear and adverse weather conditions. However, domain adaption faces two challenges: (I) it typically relies on utilizing clear image as a reference, which is challenging to obtain in practice; (II) it generally targets single adverse weather condition and performs poorly when confronting the mixture of multiple adverse weather conditions. To address these issues, we introduce a reference-free and Adverse weather condition-independent (Advent) framework (rather than a specific model architecture) that can be implemented by various backbones and heads. This is achieved by leveraging the homogeneity over short durations, getting rid of clear reference and being generalizable to arbitrary weather condition. Specifically, Advent includes three integral components: (I) Locally Sequential Mechanism (LSM) leverages temporal correlations between adjacent frames to achieve the weather-condition-agnostic effect thanks to the homogeneity behind arbitrary weather condition; (II) Globally Shuffled Mechanism (GSM) is proposed to shuffle segments processed by LSM from different positions of input sequence to prevent the overfitting to LSM-induced temporal patterns; (III) Unfolded Regularizers (URs) are the deep unfolding implementation of two proposed regularizers to penalize the model complexity to enhance across-weather generalization. We take the semantic segmentation task as an example to assess the proposed Advent framework. Extensive experiments demonstrate that the proposed Advent outperforms existing state-of-the-art baselines with large margins.

Problem

• Addresses adverse weather challenges in autonomous driving perception
• Eliminates need for clear reference images in domain adaptation
• Generalizes to mixed adverse weather conditions without overfitting

Innovation

• Leverages temporal correlation for weather-agnostic effect
• Uses shuffled segments to prevent overfitting
• Implements unfolded regularizers for generalization