PAC-Bayesian Bounds on Constrained f-Entropic Risk Measures

2025-10-13

AI Summary

Existing PAC generalization bounds—based on expected risk—fail to adequately characterize subgroup imbalance and distributional shift. To address this, we propose a constrained *f*-entropy risk measure family, enabling fine-grained control over subgroup risks via *f*-divergence. We further establish the first PAC-Bayesian analytical framework tailored to this risk measure, yielding the first class of *disentangled generalization bounds*: theoretical guarantees that separately bound both the overall risk and each subgroup’s risk. Building upon this, we design a self-bounded algorithm that jointly optimizes the PAC-Bayesian objective, incorporates *f*-divergence regularization, and models tail risk via Conditional Value-at-Risk (CVaR), directly minimizing the derived generalization upper bound. Empirical results demonstrate significant improvements in subgroup fairness and out-of-distribution robustness.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

PAC generalization bounds on the risk, when expressed in terms of the expected loss, are often insufficient to capture imbalances between subgroups in the data. To overcome this limitation, we introduce a new family of risk measures, called constrained f-entropic risk measures, which enable finer control over distributional shifts and subgroup imbalances via f-divergences, and include the Conditional Value at Risk (CVaR), a well-known risk measure. We derive both classical and disintegrated PAC-Bayesian generalization bounds for this family of risks, providing the first disintegratedPAC-Bayesian guarantees beyond standard risks. Building on this theory, we design a self-bounding algorithm that minimizes our bounds directly, yielding models with guarantees at the subgroup level. Finally, we empirically demonstrate the usefulness of our approach.

Problem

• Addressing subgroup imbalances in risk measures
• Introducing constrained f-entropic risk measures for distributional control
• Providing PAC-Bayesian guarantees beyond standard risk assessments

Innovation

• Introduces constrained f-entropic risk measures
• Derives PAC-Bayesian bounds for subgroup guarantees
• Designs self-bounding algorithm minimizing subgroup risks