Enhancing Sample Generation of Diffusion Models using Noise Level Correction

2024-12-07

AI Summary

To address the degradation in generation quality caused by misalignment between noise levels and distances to the data manifold during diffusion model denoising, this paper proposes a noise-level calibration mechanism. It explicitly models noise level as a proxy for the distance from a sample to the data manifold and introduces a lightweight, plug-and-play auxiliary correction network that dynamically refines denoising estimates at each step. The method requires no retraining of the backbone diffusion model and uniformly supports diverse restoration tasks—including image inpainting, super-resolution, deblurring, color enhancement, and compression artifact removal—while remaining compatible with mainstream samplers (e.g., DDIM). Correction subnetworks are constructed solely from pre-trained denoising networks, incorporating manifold geometric priors and task-specific constraints (e.g., masks, degradation kernels, frequency-domain restrictions). Experiments demonstrate consistent improvements in FID and LPIPS across both unconditional generation and restoration tasks, with minimal computational overhead and stable performance gains.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The denoising process of diffusion models can be interpreted as an approximate projection of noisy samples onto the data manifold. Moreover, the noise level in these samples approximates their distance to the underlying manifold. Building on this insight, we propose a novel method to enhance sample generation by aligning the estimated noise level with the true distance of noisy samples to the manifold. Specifically, we introduce a noise level correction network, leveraging a pre-trained denoising network, to refine noise level estimates during the denoising process. Additionally, we extend this approach to various image restoration tasks by integrating task-specific constraints, including inpainting, deblurring, super-resolution, colorization, and compressed sensing. Experimental results demonstrate that our method significantly improves sample quality in both unconstrained and constrained generation scenarios. Notably, the proposed noise level correction framework is compatible with existing denoising schedulers (e.g., DDIM), offering additional performance improvements.

Problem

• Diffusion Models
• Image Restoration
• Noise Reduction

Innovation

• Noise Level Correction Network
• Diffusion Model Enhancement
• Versatile Denoising Application