Removing Multiple Hybrid Adverse Weather in Video via a Unified Model

2025-03-08

AI Summary

Real-world videos under complex adverse weather conditions—such as concurrent rain, fog, and snow—exhibit heterogeneous, mixed degradations. Existing methods are constrained by single-degradation assumptions, model specificity, and the absence of paired training data for co-occurring weather phenomena. To address these limitations, we propose UniWRV, an end-to-end unified video restoration model. It introduces a weather-prior-guided module for instance-level spatial feature customization and a dynamic routing aggregation module for adaptive temporal feature fusion. Furthermore, we construct HWVideo—the first large-scale paired video dataset featuring 15 distinct mixed-weather scenarios. Extensive experiments demonstrate that UniWRV significantly outperforms degradation-specific single-weather methods across diverse multi-weather scenes, exhibits strong generalization to unseen weather combinations, and concurrently enhances performance on generic video restoration tasks.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Videos captured under real-world adverse weather conditions typically suffer from uncertain hybrid weather artifacts with heterogeneous degradation distributions. However, existing algorithms only excel at specific single degradation distributions due to limited adaption capacity and have to deal with different weather degradations with separately trained models, thus may fail to handle real-world stochastic weather scenarios. Besides, the model training is also infeasible due to the lack of paired video data to characterize the coexistence of multiple weather. To ameliorate the aforementioned issue, we propose a novel unified model, dubbed UniWRV, to remove multiple heterogeneous video weather degradations in an all-in-one fashion. Specifically, to tackle degenerate spatial feature heterogeneity, we propose a tailored weather prior guided module that queries exclusive priors for different instances as prompts to steer spatial feature characterization. To tackle degenerate temporal feature heterogeneity, we propose a dynamic routing aggregation module that can automatically select optimal fusion paths for different instances to dynamically integrate temporal features. Additionally, we managed to construct a new synthetic video dataset, termed HWVideo, for learning and benchmarking multiple hybrid adverse weather removal, which contains 15 hybrid weather conditions with a total of 1500 adverse-weather/clean paired video clips. Real-world hybrid weather videos are also collected for evaluating model generalizability. Comprehensive experiments demonstrate that our UniWRV exhibits robust and superior adaptation capability in multiple heterogeneous degradations learning scenarios, including various generic video restoration tasks beyond weather removal.

Problem

• Handling multiple hybrid adverse weather artifacts in videos
• Overcoming limitations of single-degradation models in real-world scenarios
• Addressing lack of paired video data for multiple weather conditions

Innovation

• Unified model removes multiple weather degradations
• Weather prior guided module steers spatial features
• Dynamic routing aggregation integrates temporal features