Hierarchical Bayesian Estimation of Covariance Matrices

2026-06-23

AI Summary

This work addresses the limitations of high-dimensional covariance matrix estimation, which often stems from restrictive invariance assumptions or a lack of theoretical optimality. The authors propose a hierarchical Bayesian framework leveraging the orthogonal group $O(p)$-equivariant structure and establish, for the first time, that the minimum risk estimator within the $O(p)$-equivariant class coincides with the Oracle Bayes rule under the Haar measure. They flexibly model the unknown eigenvalue distribution using a finite Pólya tree prior and implement posterior inference via Gibbs sampling. The resulting shrinkage estimator asymptotically approaches theoretical optimality under multiple loss functions. Simulations demonstrate that the method accurately recovers the spectral shape of eigenvalues and substantially outperforms classical approaches such as Ledoit–Wolf and Haff estimators, achieving performance close to that of the Oracle.

Abstract

We develop a hierarchical Bayesian framework for covariance matrix estimation built on a key observation: while equivariance under the full general linear group GL(p) is well known, it is an extremely restrictive property -- estimators equivariant to GL(p) are limited to scalar multiples of the sample covariance matrix and carry considerably larger risks than shrinkage estimators. By contrast, commonly used shrinkage estimators, including the Haff empirical Bayes estimator, and the Ledoit--Wolf estimators, are all equivariant under the smaller orthogonal group O(p). Exploiting this structure, we establish that the Haar measure Bayes rule in an oracle eigenvalue model is the minimum risk estimator within the class of O(p)-equivariant estimators, and derive oracle Bayes rules for the covariance and precision matrices under the squared Frobenius, Stein, and squared Stein loss functions. These oracle rules serve as theoretical benchmarks that dominate all commonly used estimators. To approximate them when the true eigenvalues are unknown, we introduce a hierarchical Bayes model that places a finite P'olya tree prior on the eigenvalue distribution and uses Gibbs sampling to generate posterior draws, yielding both shrinkage estimates for the eigenvalues and approximations to the oracle Bayes rules. Simulations suggest that the finite P'olya tree prior is able to recover the general form of the distribution of the eigenvalues, and confirm that the resulting estimators closely approach oracle performance, substantially outperforming classical competitors for both covariance and precision matrix estimation.

Problem

• covariance matrix estimation
• equivariance
• oracle Bayes rule
• shrinkage estimation
• eigenvalue distribution

Innovation

• hierarchical Bayesian
• O(p)-equivariance
• oracle Bayes rule
• finite Pólya tree prior
• covariance matrix estimation