UnfoldIR: Rethinking Deep Unfolding Network in Illumination Degradation Image Restoration

2025-05-10

AI Summary

To address the performance limitations of deep unfolding networks (DUNs) in illumination-degraded image restoration (IDIR), this paper proposes UnfoldIR—a novel DUN architecture. Methodologically, UnfoldIR introduces (1) a reflectance-illumination-coupled degradation model tailored for IDIR; (2) a dual-module unfolding framework featuring reflectance-assisted multi-stage illumination correction and illumination-guided reflectance enhancement; and (3) visual state space (VSS)-driven non-local illumination smoothing, frequency-aware texture enhancement, and an inter-layer information consistency self-supervised loss. Evaluated across five IDIR tasks and three downstream vision tasks, UnfoldIR consistently outperforms state-of-the-art methods, achieving significant improvements in fine-detail recovery, noise robustness, and unsupervised generalization capability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Deep unfolding networks (DUNs) are widely employed in illumination degradation image restoration (IDIR) to merge the interpretability of model-based approaches with the generalization of learning-based methods. However, the performance of DUN-based methods remains considerably inferior to that of state-of-the-art IDIR solvers. Our investigation indicates that this limitation does not stem from structural shortcomings of DUNs but rather from the limited exploration of the unfolding structure, particularly for (1) constructing task-specific restoration models, (2) integrating advanced network architectures, and (3) designing DUN-specific loss functions. To address these issues, we propose a novel DUN-based method, UnfoldIR, for IDIR tasks. UnfoldIR first introduces a new IDIR model with dedicated regularization terms for smoothing illumination and enhancing texture. We unfold the iterative optimized solution of this model into a multistage network, with each stage comprising a reflectance-assisted illumination correction (RAIC) module and an illumination-guided reflectance enhancement (IGRE) module. RAIC employs a visual state space (VSS) to extract non-local features, enforcing illumination smoothness, while IGRE introduces a frequency-aware VSS to globally align similar textures, enabling mildly degraded regions to guide the enhancement of details in more severely degraded areas. This suppresses noise while enhancing details. Furthermore, given the multistage structure, we propose an inter-stage information consistent loss to maintain network stability in the final stages. This loss contributes to structural preservation and sustains the model's performance even in unsupervised settings. Experiments verify our effectiveness across 5 IDIR tasks and 3 downstream problems.

Problem

• Improving DUN performance in illumination degradation restoration
• Enhancing unfolding structure for task-specific models and architectures
• Designing DUN-specific loss functions for stability and detail preservation

Innovation

• Introduces task-specific IDIR model with regularization
• Unfolds solution into multistage RAIC and IGRE modules
• Proposes inter-stage loss for network stability