Empirical Bayes for correlated Gaussian sequence model

2026-07-03

AI Summary

This study addresses the challenge of empirical Bayes estimation under high-dimensional Gaussian observations with dependence, moving beyond conventional independence assumptions. It proposes a novel approach for estimating the signal prior based on composite marginal likelihood (CML). The work establishes, for the first time, convergence rate theory for CML estimators in dependent Gaussian sequences, proving that they converge at the parametric rate \( n_*^{-1/2} \) under a weighted Hellinger distance, where the effective sample size is \( n_* = n / \kappa_0 \) and \( \kappa_0 \) is determined by the spectral radius of the correlation matrix. Leveraging high-dimensional auxiliary statistics and geometric Brascamp–Lieb inequalities, the method is shown to achieve near-minimax optimality. Its inferential efficacy is further demonstrated in Bayesian linear regression and nonlinear single-index models.

Abstract

Empirical Bayes methods are among the most widely used statistical methods for large-scale inference. A central paradigm is the NPMLE, whose theoretical guarantees are by now well understood for the independent Gaussian sequence model. In this paper, we study empirical Bayes estimation from dependent observations in the Gaussian sequence model. We show that the maximum Composite Marginal Likelihood (CML) estimator, which ignores all correlations in the likelihood, converges in weighted Hellinger distance at the rate $n_*^{-1/2}$, where $n_*=n/κ_0$ is the `effective sample size' determined solely by the number of observations $n$ and the spectral radius $κ_0$ of the correlation matrix of the Gaussian observations. A complementary minimax lower bound shows that $n_*$ indeed serves as the right complexity measure, and that the CML estimator is nearly rate optimal under general dependence. We consider two concrete applications. In the first, we consider Bayesian linear regression, where the signal prior is estimated via CML applied to the least squares estimator. In the second, we consider the more challenging Bayesian nonlinear single-index model, where prior is estimated by CML applied to a one-step debiased gradient descent. In both applications, although the full likelihood landscape can be arbitrarily complicated and intractable, our CML method is facilitated by exploiting the high-dimensional distribution of the auxiliary statistics through a correlated Gaussian sequence model. The key ingredient in the proof of our results is a sharp local maximal inequality for the log composite marginal likelihood process under dependent Gaussian observations. In contrast to standard empirical process methods, we prove this inequality by leveraging a recent geometric Brascamp-Lieb inequality for Gaussian measures.

Problem

• Empirical Bayes
• correlated Gaussian sequence model
• dependent observations
• effective sample size
• composite marginal likelihood

Innovation

• Empirical Bayes
• Composite Marginal Likelihood
• Correlated Gaussian sequence model
• Effective sample size
• Brascamp-Lieb inequality