Problem
Research questions and friction points this paper is trying to address.
adverse weather removal
image restoration
multi-weather conditions
computational efficiency
parameter efficiency
WeatherRemover: All-in-one Adverse Weather Removal with Multi-scale Feature Map Compression
🤖 AI Summary
This work addresses the challenge of image degradation under adverse weather conditions—such as rain, snow, and fog—which cause blurriness, occlusion, and low brightness, severely impairing downstream vision tasks. Existing approaches are often limited to a single weather type or incur excessive computational costs. To overcome these limitations, the authors propose a lightweight, unified restoration model built upon a U-Net-like architecture that integrates a multi-scale pyramid Vision Transformer, gated mechanisms, and channel attention modules. A linear spatial compression strategy is further introduced to efficiently capture long-range dependencies. The proposed method achieves significant reductions in model parameters, inference time, and memory consumption while maintaining superior restoration performance across multiple weather types, outperforming current state-of-the-art multi-weather image restoration approaches.
📝 Abstract
Photographs taken in adverse weather conditions often suffer from blurriness, occlusion, and low brightness due to interference from rain, snow, and fog. These issues can significantly hinder the performance of subsequent computer vision tasks, making the removal of weather effects a crucial step in image enhancement. Existing methods primarily target specific weather conditions, with only a few capable of handling multiple weather scenarios. However, mainstream approaches often overlook performance considerations, resulting in large parameter sizes, long inference times, and high memory costs. In this study, we introduce the WeatherRemover model, designed to enhance the restoration of images affected by various weather conditions while balancing performance. Our model adopts a UNet-like structure with a gating mechanism and a multi-scale pyramid vision Transformer. It employs channel-wise attention derived from convolutional neural networks to optimize feature extraction, while linear spatial reduction helps curtail the computational demands of attention. The gating mechanisms, strategically placed within the feed-forward and downsampling phases, refine the processing of information by selectively addressing redundancy and mitigating its influence on learning. This approach facilitates the adaptive selection of essential data, ensuring superior restoration and maximizing efficiency. Additionally, our lightweight model achieves an optimal balance between restoration quality, parameter efficiency, computational overhead, and memory usage, distinguishing it from other multi-weather models, thereby meeting practical application demands effectively. The source code is available at https://github.com/RICKand-MORTY/WeatherRemover.
Innovation
Methods, ideas, or system contributions that make the work stand out.
multi-scale feature compression
gating mechanism
pyramid vision Transformer
channel-wise attention
lightweight weather removal
🔎 Similar Papers
No similar papers found.
