Bayes Estimators with Performance Comparable to Empirical Bayes Estimators and Improved Local Robustness

2026-09-06

AI Summary

本文针对线性回归模型中经验贝叶斯估计器对超参数扰动敏感的问题，提出了一种具有相同超额均方误差但局部鲁棒性更好的广义贝叶斯估计方法。

Abstract

Bayes estimation has been extensively studied and widely used in statistics, decision theory, signal processing, machine learning, and system identification. Among its variants, empirical Bayes (EB) estimation has attracted considerable attention due to its favorable estimation performance and computational tractability. However, the direct plug-in dependence of an EB estimator on hyperparameters can make it locally sensitive to hyper-parameter perturbations. This paper considers the linear regression model and focuses on the EB estimator by employing the marginal maximum likelihood hyper-parameter estimator. For conciseness, this estimator is simply referred to as the EB estimator. Given a family of EB weighting functions, a generalized Bayes estimator is constructed with the same excess mean squared error (XMSE) as the corresponding EB estimator. Here, the XMSE is a second-order asymptotic measure of the mean squared error difference between the estimator of interest and the maximum likelihood estimator. Furthermore, the EB estimator is shown to be at most firstorder sensitive to hyper-parameter perturbations, whereas the constructed Bayes estimator is at most second-order sensitive, making it locally more robust. The computational complexities of these two estimators are also analyzed. In some cases, the constructed Bayes estimator can be computationally comparable to, or more efficient than, the EB estimator. These theoretical results are further supported by numerical simulations.

Problem

• Empirical Bayes
• local robustness
• hyper-parameter perturbations
• sensitivity

Innovation

• Bayes Estimator
• Empirical Bayes
• XMSE
• Local Robustness
• Hyper-parameter Sensitivity