Taming diffusion models for image restoration: a review

2024-09-16

Philosophical transactions. Series A, Mathematical, physical, and engineering sciences

AI Summary

This paper systematically surveys diffusion models for low-level vision restoration tasks—including image denoising, deblurring, and dehazing—addressing three core challenges: low sampling efficiency, fidelity-diversity trade-off imbalance, and difficulty in incorporating physical priors. Methodologically, it introduces the first taxonomy of diffusion modeling paradigms tailored to image restoration, unifying approaches based on Denoising Diffusion Probabilistic Models (DDPM), stochastic differential equation (SDE) solvers, conditional guidance, and Bayesian inverse problem modeling. The survey critically analyzes state-of-the-art methods, identifying key limitations in inference speed, reconstruction accuracy, and interpretability. Its principal contributions include: (i) a structured classification framework that clarifies design principles and inter-paradigm relationships; and (ii) two forward-looking research directions—task-adaptive sampling strategies and interpretable, physics-informed diffusion modeling—thereby establishing both a theoretical foundation and practical guidelines for diffusion-driven image restoration.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Diffusion models (DMs) have achieved remarkable progress in generative modelling, particularly in enhancing image quality to conform to human preferences. Recently, these models have also been applied to low-level computer vision for photo-realistic image restoration (IR) in tasks such as image denoising, deblurring and dehazing. In this review, we introduce key constructions in DMs and survey contemporary techniques that make use of DMs in solving general IR tasks. We also point out the main challenges and limitations of existing diffusion-based IR frameworks and provide potential directions for future work. This article is part of the theme issue ‘Generative modelling meets Bayesian inference: a new paradigm for inverse problems’.

Problem

• Surveying diffusion models for image restoration tasks
• Addressing challenges in diffusion-based image reconstruction
• Exploring applications in denoising deblurring and dehazing

Innovation

• Diffusion models enhance image quality restoration
• Survey techniques for general image restoration tasks
• Address challenges in diffusion-based restoration frameworks