Machine Learning. The Science of Selection under Uncertainty

2025-09-25

AI Summary

This paper addresses the uncertainty-aware model selection problem in machine learning—specifically, how to robustly select a predictor with superior generalization performance from a candidate set, under noise induced by stochastic data sampling. We propose a unified theoretical framework that systematically integrates classical concentration inequalities (Markov, Chebyshev, Hoeffding, Bernstein) with VC-dimension analysis, Occam’s razor principles, and PAC-Bayes bounds, extending them to weighted majority voting and online learning settings. For the first time, we simultaneously derive tight generalization error bounds and regret upper bounds in both offline and online regimes, accommodating both stochastic and adversarial environments. The framework provides a statistically rigorous, broadly applicable, and quantitatively tight foundation for model selection in supervised learning and sequential decision-making.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising

Abstract

Learning, whether natural or artificial, is a process of selection. It starts with a set of candidate options and selects the more successful ones. In the case of machine learning the selection is done based on empirical estimates of prediction accuracy of candidate prediction rules on some data. Due to randomness of data sampling the empirical estimates are inherently noisy, leading to selection under uncertainty. The book provides statistical tools to obtain theoretical guarantees on the outcome of selection under uncertainty. We start with concentration of measure inequalities, which are the main statistical instrument for controlling how much an empirical estimate of expectation of a function deviates from the true expectation. The book covers a broad range of inequalities, including Markov's, Chebyshev's, Hoeffding's, Bernstein's, Empirical Bernstein's, Unexpected Bernstein's, kl, and split-kl. We then study the classical (offline) supervised learning and provide a range of tools for deriving generalization bounds, including Occam's razor, Vapnik-Chervonenkis analysis, and PAC-Bayesian analysis. The latter is further applied to derive generalization guarantees for weighted majority votes. After covering the offline setting, we turn our attention to online learning. We present the space of online learning problems characterized by environmental feedback, environmental resistance, and structural complexity. A common performance measure in online learning is regret, which compares performance of an algorithm to performance of the best prediction rule in hindsight, out of a restricted set of prediction rules. We present tools for deriving regret bounds in stochastic and adversarial environments, and under full information and bandit feedback.

Problem

• Developing statistical guarantees for machine learning under data uncertainty
• Providing generalization bounds for offline supervised learning algorithms
• Deriving regret bounds for online learning in various environmental settings

Innovation

• Statistical tools for selection under uncertainty guarantees
• Generalization bounds via Occam's razor and PAC-Bayesian analysis
• Regret bounds for online learning in adversarial environments