Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions

2025-11-12

AI Summary

This work addresses the problem of recovering the underlying distribution of multidimensional signals corrupted by additive noise with known magnitude but unknown distribution. To overcome the limitation of conventional methods that rely on explicit noise priors, we propose a universal denoiser independent of both signal and noise distributions. Our method leverages optimal transport theory: it estimates the score (log-density gradient) and its higher-order derivatives via score matching, then constructs low-shrinkage operators (T_1) and (T_2) to approximately solve the Monge–Ampère equation. Compared to the Bayesian optimal denoiser based on Tweedie’s formula, our approach achieves a qualitative leap in distributional fidelity—achieving (O(sigma^4)) accuracy for generalized moments and (O(sigma^6)) for density approximation. Experiments demonstrate robustness across diverse signal-noise distribution pairs and consistently superior distribution recovery performance over state-of-the-art methods.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We revisit the problem of denoising from noisy measurements where only the noise level is known, not the noise distribution. In multi-dimensions, independent noise $Z$ corrupts the signal $X$, resulting in the noisy measurement $Y = X + sigma Z$, where $sigma in (0, 1)$ is a known noise level. Our goal is to recover the underlying signal distribution $P_X$ from denoising $P_Y$. We propose and analyze universal denoisers that are agnostic to a wide range of signal and noise distributions. Our distributional denoisers offer order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie's formula, if the focus is on the entire distribution $P_X$ rather than on individual realizations of $X$. Our denoisers shrink $P_Y$ toward $P_X$ optimally, achieving $O(sigma^4)$ and $O(sigma^6)$ accuracy in matching generalized moments and density functions. Inspired by optimal transport theory, the proposed denoisers are optimal in approximating the Monge-Amp`ere equation with higher-order accuracy, and can be implemented efficiently via score matching. Let $q$ represent the density of $P_Y$; for optimal distributional denoising, we recommend replacing the Bayes-optimal denoiser, [ mathbf{T}^*(y) = y + sigma^2 abla log q(y), ] with denoisers exhibiting less aggressive distributional shrinkage, [ mathbf{T}_1(y) = y + frac{sigma^2}{2} abla log q(y), ] [ mathbf{T}_2(y) = y + frac{sigma^2}{2} abla log q(y) - frac{sigma^4}{8} abla left( frac{1}{2} | abla log q(y) |^2 + abla cdot abla log q(y) ight) . ]

Problem

• Denoising signals from noisy measurements with known noise level only
• Developing universal denoisers agnostic to signal and noise distributions
• Improving distribution recovery accuracy over traditional Bayes-optimal denoisers

Innovation

• Universal denoisers agnostic to signal and noise distributions
• Optimal distributional shrinkage via modified Tweedie formula
• Higher-order accuracy in density approximation via score matching