Diffusion Image Prior

2025-03-27

AI Summary

Addressing the challenge of explicitly modeling unknown degradation processes in real-world image restoration, this paper proposes a zero-shot blind restoration method that requires no prior knowledge of the degradation. The core innovation lies in the first integration of a pre-trained diffusion model into the deep image prior (DIP) framework, enabling clean image reconstruction directly via latent-space optimization with an early-stopping strategy—without specifying the degradation type. This paradigm eliminates reliance on parameterized degradation models, significantly broadening applicability to diverse real-world scenarios. Extensive experiments demonstrate state-of-the-art performance across multiple blind restoration tasks—including JPEG artifact removal, waterdrop distortion correction, denoising, and super-resolution—validating the method’s strong generalization capability and robustness under unknown degradations.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Zero-shot image restoration (IR) methods based on pretrained diffusion models have recently achieved significant success. These methods typically require at least a parametric form of the degradation model. However, in real-world scenarios, the degradation may be too complex to define explicitly. To handle this general case, we introduce the Diffusion Image Prior (DIIP). We take inspiration from the Deep Image Prior (DIP)[16], since it can be used to remove artifacts without the need for an explicit degradation model. However, in contrast to DIP, we find that pretrained diffusion models offer a much stronger prior, despite being trained without knowledge from corrupted data. We show that, the optimization process in DIIP first reconstructs a clean version of the image before eventually overfitting to the degraded input, but it does so for a broader range of degradations than DIP. In light of this result, we propose a blind image restoration (IR) method based on early stopping, which does not require prior knowledge of the degradation model. We validate DIIP on various degradation-blind IR tasks, including JPEG artifact removal, waterdrop removal, denoising and super-resolution with state-of-the-art results.

Problem

• Handles complex undefined real-world image degradations
• Proposes blind restoration without degradation model knowledge
• Improves over DIP using stronger diffusion-based priors

Innovation

• Uses pretrained diffusion models as strong prior
• Proposes early stopping for blind image restoration
• Handles complex degradations without explicit models