Uncertainty-guided Perturbation for Image Super-Resolution Diffusion Model

2025-03-24

AI Summary

To address insufficient utilization of low-resolution (LR) information in diffusion-based image super-resolution, this paper proposes a region-adaptive noise modulation mechanism grounded in pixel-wise uncertainty estimation: noise weights are reduced in flat regions to preserve LR details and increased along texture edges to enhance reconstruction fidelity. This work pioneers the integration of uncertainty modeling into the super-resolution diffusion process, enabling Uncertainty-Guided Noise Weighting (UGNW). We further design a lightweight UPSR network that synergistically fuses multi-scale features with an enhanced conditional U-Net backbone. Extensive experiments on standard benchmarks—including Set5 and DIV2K—demonstrate that our method surpasses state-of-the-art approaches in both PSNR and SSIM. Moreover, it achieves a 23% speedup in inference latency and reduces model parameters by 18%, while significantly improving texture sharpness and structural consistency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Diffusion-based image super-resolution methods have demonstrated significant advantages over GAN-based approaches, particularly in terms of perceptual quality. Building upon a lengthy Markov chain, diffusion-based methods possess remarkable modeling capacity, enabling them to achieve outstanding performance in real-world scenarios. Unlike previous methods that focus on modifying the noise schedule or sampling process to enhance performance, our approach emphasizes the improved utilization of LR information. We find that different regions of the LR image can be viewed as corresponding to different timesteps in a diffusion process, where flat areas are closer to the target HR distribution but edge and texture regions are farther away. In these flat areas, applying a slight noise is more advantageous for the reconstruction. We associate this characteristic with uncertainty and propose to apply uncertainty estimate to guide region-specific noise level control, a technique we refer to as Uncertainty-guided Noise Weighting. Pixels with lower uncertainty (i.e., flat regions) receive reduced noise to preserve more LR information, therefore improving performance. Furthermore, we modify the network architecture of previous methods to develop our Uncertainty-guided Perturbation Super-Resolution (UPSR) model. Extensive experimental results demonstrate that, despite reduced model size and training overhead, the proposed UWSR method outperforms current state-of-the-art methods across various datasets, both quantitatively and qualitatively.

Problem

• Enhancing LR information utilization in diffusion-based super-resolution
• Guiding noise levels by uncertainty in different image regions
• Improving performance with reduced model size and training cost

Innovation

• Uncertainty-guided region-specific noise control
• Modified network architecture for UPSR model
• Utilizes LR information via uncertainty weighting