Characterizing Online and Private Learnability under Distributional Constraints via Generalized Smoothness

2026-02-24

AI Summary

This work investigates the learnability of online and differentially private learning in the presence of an adaptive adversary who can choose data-generating distributions from a given family. By introducing a notion of generalized smoothness, the study unifies and characterizes the necessary and sufficient conditions for learnability under distributional constraints in both online and private settings, establishing this property as the central criterion for learnability in these regimes. The approach integrates VC dimension theory, combinatorial parameters (specifically, the fragmentation number), and differential privacy techniques to design a general low-regret algorithm that requires no prior knowledge of the distribution family. Key contributions include deriving VC-dimension-dependent regret bounds for any hypothesis class of finite VC dimension under generalized smooth distribution families, and providing a complete characterization of private learnability under distributional constraints.

Technology Category

• Machine Learning: Online Learning & Bandits
• Game Theory and Economic Paradigms: Adversarial Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Understanding minimal assumptions that enable learning and generalization is perhaps the central question of learning theory. Several celebrated results in statistical learning theory, such as the VC theorem and Littlestone's characterization of online learnability, establish conditions on the hypothesis class that allow for learning under independent data and adversarial data, respectively. Building upon recent work bridging these extremes, we study sequential decision making under distributional adversaries that can adaptively choose data-generating distributions from a fixed family $U$ and ask when such problems are learnable with sample complexity that behaves like the favorable independent case. We provide a near complete characterization of families $U$ that admit learnability in terms of a notion known as generalized smoothness i.e. a distribution family admits VC-dimension-dependent regret bounds for every finite-VC hypothesis class if and only if it is generalized smooth. Further, we give universal algorithms that achieve low regret under any generalized smooth adversary without explicit knowledge of $U$. Finally, when $U$ is known, we provide refined bounds in terms of a combinatorial parameter, the fragmentation number, that captures how many disjoint regions can carry nontrivial mass under $U$. These results provide a nearly complete understanding of learnability under distributional adversaries. In addition, building upon the surprising connection between online learning and differential privacy, we show that the generalized smoothness also characterizes private learnability under distributional constraints.

Problem

• distributional constraints
• online learnability
• private learnability
• generalized smoothness
• sample complexity

Innovation

• generalized smoothness
• distributional adversaries
• online learnability
• private learnability
• fragmentation number