Whitened Score Diffusion: A Structured Prior for Imaging Inverse Problems

2025-05-15

AI Summary

Conventional score-based diffusion models suffer from training instability under anisotropic Gaussian noise, as denoising score matching requires explicit inversion of the covariance matrix. Method: We propose the Whitened Score Diffusion Model (WSDM), which introduces a whitening transformation in the forward process to recast any Gaussian degradation as standard Brownian motion—thereby eliminating the need for covariance inversion and enabling stable training. Contribution/Results: Theoretically, we establish, for the first time, the equivalence between whitened scores and general Gaussian forward processes. Methodologically, WSDM supports customizable spectral priors, providing strong Bayesian regularization for imaging inverse problems under structured noise. Empirically, on deblurring and super-resolution tasks using CIFAR-10 and CelebA, WSDM significantly outperforms isotropic diffusion priors in reconstruction quality, demonstrating superior robustness and efficacy under realistic, non-stationary degradation scenarios.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Conventional score-based diffusion models (DMs) may struggle with anisotropic Gaussian diffusion processes due to the required inversion of covariance matrices in the denoising score matching training objective cite{vincent_connection_2011}. We propose Whitened Score (WS) diffusion models, a novel SDE-based framework that learns the Whitened Score function instead of the standard score. This approach circumvents covariance inversion, extending score-based DMs by enabling stable training of DMs on arbitrary Gaussian forward noising processes. WS DMs establish equivalence with FM for arbitrary Gaussian noise, allow for tailored spectral inductive biases, and provide strong Bayesian priors for imaging inverse problems with structured noise. We experiment with a variety of computational imaging tasks using the CIFAR and CelebA ($64 imes64$) datasets and demonstrate that WS diffusion priors trained on anisotropic Gaussian noising processes consistently outperform conventional diffusion priors based on isotropic Gaussian noise.

Problem

• Avoids covariance inversion in diffusion models
• Enables stable training with arbitrary Gaussian noise
• Improves performance in imaging inverse problems

Innovation

• Whitened Score diffusion avoids covariance inversion
• Enables stable training with arbitrary Gaussian noise
• Provides strong Bayesian priors for imaging