When No-Reference Image Quality Models Meet MAP Estimation in Diffusion Latents

2024-03-11

AI Summary

To address the challenges of unknown distortion types and reference-dependent quality assessment in real-world image enhancement, this paper pioneers the integration of no-reference image quality assessment (NR-IQA) models as natural image priors into a diffusion-based maximum a posteriori (MAP) estimation framework. Methodologically, it performs perception-driven, end-to-end inversion and enhancement via NR-IQA-guided gradient ascent within a pre-trained, differentiable bijective diffusion latent space. Key contributions include: (1) establishing the first NR-IQA prior modeling paradigm for MAP optimization in diffusion latent spaces; (2) proposing a novel, perception-oriented evaluation criterion for NR-IQA models—emphasizing perceptual optimization capability over statistical correlation; and (3) designing a multi-NR-IQA prior fusion mechanism that significantly improves robustness to complex, mixed distortions. Experiments demonstrate that the method achieves superior fidelity and human-perceived quality under unknown distortions, outperforming state-of-the-art reference-based and no-reference enhancement approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: State Estimation
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Contemporary no-reference image quality assessment (NR-IQA) models can effectively quantify perceived image quality, often achieving strong correlations with human perceptual scores on standard IQA benchmarks. Yet, limited efforts have been devoted to treating NR-IQA models as natural image priors for real-world image enhancement, and consequently comparing them from a perceptual optimization standpoint. In this work, we show -- for the first time -- that NR-IQA models can be plugged into the maximum a posteriori (MAP) estimation framework for image enhancement. This is achieved by performing gradient ascent in the diffusion latent space rather than in the raw pixel domain, leveraging a pretrained differentiable and bijective diffusion process. Likely, different NR-IQA models lead to different enhanced outputs, which in turn provides a new computational means of comparing them. Unlike conventional correlation-based measures, our comparison method offers complementary insights into the respective strengths and weaknesses of the competing NR-IQA models in perceptual optimization scenarios. Additionally, we aim to improve the best-performing NR-IQA model in diffusion latent MAP estimation by incorporating the advantages of other top-performing methods. The resulting model delivers noticeably better results in enhancing real-world images afflicted by unknown and complex distortions, all preserving a high degree of image fidelity.

Problem

• Image Quality Assessment
• No-Reference IQA Models
• Image Enhancement

Innovation

• No-Reference Image Quality Assessment
• Diffusion Latent Space
• Enhanced Image Clarity