Constrained Denoising, Empirical Bayes, and Optimal Transport

2025-06-11

AI Summary

Existing Bayesian and empirical Bayesian denoisers suffer from excessive shrinkage of latent variables, particularly under misspecified priors. Method: This paper proposes a general constrained denoising framework grounded in optimal transport theory. It first characterizes, for the first time, the oracle denoiser under variance, distributional, and generalized constraints. Building on this, it introduces a modular construction that embeds arbitrary unconstrained empirical Bayes estimators into constrained counterparts, with provable convergence and minimax-optimal rates. Contribution/Results: The theoretical analysis transcends prior work—limited to variance constraints—by accommodating broader constraint classes with higher precision. Empirical validation demonstrates effectiveness in modeling chemical abundances of red giant stars and evaluating cross-league batting abilities of rookie baseball players, achieving both statistical rigor and practical interpretability.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Optimization
• Machine Learning: Bayesian Learning
• Search and Optimization: Non-convex Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

In the statistical problem of denoising, Bayes and empirical Bayes methods can"overshrink"their output relative to the latent variables of interest. This work is focused on constrained denoising problems which mitigate such phenomena. At the oracle level, i.e., when the latent variable distribution is assumed known, we apply tools from the theory of optimal transport to characterize the solution to (i) variance-constrained, (ii) distribution-constrained, and (iii) general-constrained denoising problems. At the empirical level, i.e., when the latent variable distribution is not known, we use empirical Bayes methodology to estimate these oracle denoisers. Our approach is modular, and transforms any suitable (unconstrained) empirical Bayes denoiser into a constrained empirical Bayes denoiser. We prove explicit rates of convergence for our proposed methodologies, which both extend and sharpen existing asymptotic results that have previously considered only variance constraints. We apply our methodology in two applications: one in astronomy concerning the relative chemical abundances in a large catalog of red-clump stars, and one in baseball concerning minor- and major league batting skill for rookie players.

Problem

• Mitigate overshrinkage in Bayes denoising methods
• Solve constrained denoising using optimal transport
• Estimate latent variables with empirical Bayes

Innovation

• Optimal transport for constrained denoising
• Empirical Bayes to estimate oracle denoisers
• Modular transformation of unconstrained denoisers