Exploiting Diffusion Prior for Task-driven Image Restoration

2025-07-30

AI Summary

Low-quality inputs caused by multi-factor complex degradation severely hinder the performance of task-driven image restoration (TDIR). Method: This paper introduces diffusion priors into TDIR for the first time. We propose a pixel-error-based pre-restoration strategy, feeding a noisy error map into a diffusion model and designing a lightweight, few-step denoising mechanism to accurately reconstruct task-relevant semantic details under limited computational overhead. Contribution/Results: The method achieves a superior trade-off between visual plausibility and downstream task performance. Experiments demonstrate significant improvements in both image reconstruction quality and high-level task accuracy—including classification and object detection—across diverse severe degradation scenarios. These results validate the effectiveness and practicality of diffusion priors in task-oriented image restoration.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Task-driven image restoration (TDIR) has recently emerged to address performance drops in high-level vision tasks caused by low-quality (LQ) inputs. Previous TDIR methods struggle to handle practical scenarios in which images are degraded by multiple complex factors, leaving minimal clues for restoration. This motivates us to leverage the diffusion prior, one of the most powerful natural image priors. However, while the diffusion prior can help generate visually plausible results, using it to restore task-relevant details remains challenging, even when combined with recent TDIR methods. To address this, we propose EDTR, which effectively harnesses the power of diffusion prior to restore task-relevant details. Specifically, we propose directly leveraging useful clues from LQ images in the diffusion process by generating from pixel-error-based pre-restored LQ images with mild noise added. Moreover, we employ a small number of denoising steps to prevent the generation of redundant details that dilute crucial task-related information. We demonstrate that our method effectively utilizes diffusion prior for TDIR, significantly enhancing task performance and visual quality across diverse tasks with multiple complex degradations.

Problem

• Address performance drops in high-level vision tasks due to low-quality inputs
• Handle images degraded by multiple complex factors with minimal restoration clues
• Restore task-relevant details using diffusion prior without generating redundant information

Innovation

• Utilizes diffusion prior for task-driven restoration
• Generates from pre-restored LQ images with noise
• Employs minimal denoising to preserve task details