Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining

2025-02-18

AI Summary

Low-level vision models exhibit poor generalization to unseen degradations in real-world scenarios, as existing approaches tend to overfit degradation patterns rather than learn intrinsic image content. This paper takes image deraining as a representative task and proposes a content-centric training paradigm: (i) decoupling degradation and content representations, (ii) actively controlling data complexity during training, and (iii) leveraging content priors extracted from generative models (e.g., Diffusion models or VAEs) for guidance. We systematically uncover, for the first time, the balancing mechanism between degradation characteristics and background complexity—and demonstrate its decisive role in model generalization. Extensive experiments show substantial improvements in cross-degradation generalization: on unseen rain patterns and noise types, our method achieves an average PSNR gain of 2.1 dB and reduces generalization error by 37%, outperforming prior methods in both deraining and denoising tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Personalized content ranking and presentation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Generalization remains a significant challenge for low-level vision models, which often struggle with unseen degradations in real-world scenarios despite their success in controlled benchmarks. In this paper, we revisit the generalization problem in low-level vision models. Image deraining is selected as a case study due to its well-defined and easily decoupled structure, allowing for more effective observation and analysis. Through comprehensive experiments, we reveal that the generalization issue is not primarily due to limited network capacity but rather the failure of existing training strategies, which leads networks to overfit specific degradation patterns. Our findings show that guiding networks to focus on learning the underlying image content, rather than the degradation patterns, is key to improving generalization. We demonstrate that balancing the complexity of background images and degradations in the training data helps networks better fit the image distribution. Furthermore, incorporating content priors from pre-trained generative models significantly enhances generalization. Experiments on both image deraining and image denoising validate the proposed strategies. We believe the insights and solutions will inspire further research and improve the generalization of low-level vision models.

Problem

• Generalization of low-level vision models
• Overfitting to specific degradation patterns
• Improving generalization with content-focused learning

Innovation

• Focus on image content learning
• Balance training data complexity
• Incorporate pre-trained generative priors