A theoretical framework for M-posteriors: frequentist guarantees and robustness properties

2025-10-01

AI Summary

This paper addresses the lack of frequentist guarantees and robustness theory for generalized posteriors (M-posteriors). Methodologically, it introduces novel definitions of the posterior influence function and posterior breakdown point, and conducts asymptotic analysis integrating M-estimation loss functions with prior conditions. Under mild regularity assumptions, it establishes that M-posteriors are asymptotically normal, concentrate around the corresponding M-estimator, and simultaneously achieve frequentist consistency and Bayesian interpretability. The key contributions are: (i) the first systematic robustness characterization of M-posteriors, formalized via influence functions and breakdown points; (ii) a natural extension of classical M-estimation robustness to Bayesian posterior inference; and (iii) theoretical validity across broad settings—including generalized linear models and quantile regression—supported by numerical experiments demonstrating stability under outliers and model misspecification.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Intelligent Robots: State Estimation

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Large-scale security measurements

Abstract

We provide a theoretical framework for a wide class of generalized posteriors that can be viewed as the natural Bayesian posterior counterpart of the class of M-estimators in the frequentist world. We call the members of this class M-posteriors and show that they are asymptotically normally distributed under mild conditions on the M-estimation loss and the prior. In particular, an M-posterior contracts in probability around a normal distribution centered at an M-estimator, showing frequentist consistency and suggesting some degree of robustness depending on the reference M-estimator. We formalize the robustness properties of the M-posteriors by a new characterization of the posterior influence function and a novel definition of breakdown point adapted for posterior distributions. We illustrate the wide applicability of our theory in various popular models and illustrate their empirical relevance in some numerical examples.

Problem

• Theoretical framework for M-posteriors with frequentist guarantees
• Characterizing robustness through posterior influence functions
• Establishing asymptotic normality under mild conditions

Innovation

• Theoretical framework for M-posteriors with frequentist guarantees
• M-posteriors contract around M-estimators showing robustness
• New characterization of posterior influence function and breakdown point