Robust Bayes-assisted Confidence Regions

2024-10-26

AI Summary

This paper addresses the challenge of simultaneously achieving accurate frequentist coverage and effective incorporation of prior information in constructing parameter confidence regions. Method: We propose Frequency-Adaptive Bayesian Confidence Regions (FAB-CRs), built upon natural exponential-family likelihoods and analytically tractable shrinkage priors—such as horseshoe-type priors—that satisfy polynomial-tail conditions. We establish, for the first time, that the posterior mean always lies within the FAB-CR and prove its boundedness under extreme observations and asymptotic frequentist calibration. Results: The resulting confidence regions achieve exact 1−α frequentist coverage for both Gaussian and general natural exponential-family models; moreover, intervals tighten adaptively for parameters with higher posterior density. The associated FAB estimator coincides with classical Bayesian shrinkage estimators, thereby establishing, within a unified framework, the theoretical equivalence between robust frequentist inference and Bayesian shrinkage.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Probabilistic Inference
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

The Frequentist, Assisted by Bayes (FAB) framework aims to construct confidence regions that leverage information about parameter values in the form of a prior distribution. FAB confidence regions (FAB-CRs) have smaller volume for values of the parameter that are likely under the prior, while maintaining exact frequentist coverage. This work introduces several methodological and theoretical contributions to the FAB framework. For Gaussian likelihoods, we show that the posterior mean of the parameter of interest is always contained in the FAB-CR. As such, the posterior mean constitutes a natural notion of FAB estimator to be reported alongside the FAB-CR. More generally, we show that for a likelihood in the natural exponential family, a transformation of the posterior mean of the natural parameter is always contained in the FAB-CR. For Gaussian likelihoods, we show that power law tails conditions on the marginal likelihood induce robust FAB-CRs, that are uniformly bounded and revert to the standard frequentist confidence intervals for extreme observations. We translate this result into practice by proposing a class of shrinkage priors for the FAB framework that satisfy this condition without sacrificing analytical tractability. The resulting FAB estimators are equal to prominent Bayesian shrinkage estimators, including the horseshoe estimator, thereby establishing insightful connections between robust FAB-CRs and Bayesian shrinkage methods.

Problem

• Construct confidence regions using prior information efficiently
• Extend FAB framework to natural exponential family likelihoods
• Develop robust bounded-influence priors for FAB confidence regions

Innovation

• FAB framework leverages prior information for confidence regions
• Posterior mean contained in FAB-CR for Gaussian likelihoods
• Power-law tail conditions induce robust bounded FAB-CRs