Sharp regret-Hellinger bounds for Gaussian empirical Bayes via polynomial approximation

2026-05-03

AI Summary

This work investigates the direct relationship between the regret (excess risk) of unregularized Bayes rules and the Hellinger distance of marginal densities within a Gaussian empirical Bayes framework. By developing an analytical framework that avoids recursive regularization and leveraging polynomial approximation together with Bernstein-type inequalities in weighted L² spaces, the authors establish sharp, non-asymptotic bounds linking these two quantities. This approach yields sharper—sometimes minimax optimal—regret bounds and clarifies the necessity of regularization under heavy-tailed priors. Specifically, for compactly supported priors, a regret upper bound of order \(O(\varepsilon^2 \log(1/\varepsilon)/\log\log(1/\varepsilon))\) is established; improved bounds are also derived for exponential-tail priors, leading to enhanced regret performance for nonparametric maximum likelihood estimation.

Abstract

A central problem in the theory of empirical Bayes is to control the regret (excess risk) of a learned Bayes rule by the Hellinger distance between the estimated and true marginal densities. In the normal means model, the classical result of Jiang and Zhang (2009, Annals of Statistics) achieves this only after regularizing the Bayes rule and incurs an extraneous cubic logarithmic factor through a delicate recursive argument. This paper introduces a new technique, based on polynomial approximation and Bernstein-type inequalities for weighted $L_2$ norms, that bounds the unregularized regret directly. The method is conceptually simpler and yields sharper, sometimes optimal, regret bounds. For compactly supported priors, we prove the sharp bound that the regret is at most $O(ε^2 \log(1/ε)/\log\log(1/ε))$, where $ε$ is the Hellinger distance between the marginal densities. The same method also extends to priors with exponential tails. Conversely, we show that regularization is genuinely necessary for heavy-tailed priors under only bounded moment assumptions. As a statistical consequence, we obtain improved regret bounds for the nonparametric maximum likelihood estimator.

Problem

• empirical Bayes
• regret
• Hellinger distance
• Gaussian model
• excess risk

Innovation

• polynomial approximation
• Bernstein inequality
• empirical Bayes
• regret bound
• Hellinger distance