The Empirical Mean is Minimax Optimal for Local Glivenko-Cantelli

2024-10-02

AI Summary

This paper investigates the optimality of the empirical mean estimator (EME) within the Local Glivenko–Cantelli (LGC) framework: Can any estimator outperform EME—either by learning broader classes of probability distributions or achieving faster risk decay rates? Leveraging distribution-dependent uniform convergence analysis, minimax risk theory, geometric characterization of measure spaces, and empirical process theory, we establish, for the first time, that EME is minimax optimal under reasonable constraints excluding pathological infinite-dimensional structures. We prove that expanding the learnable distribution class and accelerating convergence rates are fundamentally incompatible objectives. We derive necessary and sufficient conditions for strictly enlarging the class of learnable measures. Furthermore, we construct tight risk upper bounds and explicit counterexamples, systematically characterizing how pathological geometric structures decisively determine the boundary of learnability.

Technology Category

• Machine Learning: Learning Theory
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We revisit the recently introduced Local Glivenko-Cantelli setting, which studies distribution-dependent uniform convergence rates of the Empirical Mean Estimator (EME). In this work, we investigate generalizations of this setting where arbitrary estimators are allowed rather than just the EME. Can a strictly larger class of measures be learned? Can better risk decay rates be obtained? We provide exhaustive answers to these questions, which are both negative, provided the learner is barred from exploiting some infinite-dimensional pathologies. On the other hand, allowing such exploits does lead to a strictly larger class of learnable measures.

Problem

• Generalize Local Glivenko-Cantelli setting beyond Empirical Mean Estimator
• Determine if larger measure classes or better risk rates are achievable
• Assess impact of infinite-dimensional pathologies on learnability

Innovation

• Generalizes Local Glivenko-Cantelli with arbitrary estimators
• Proves no better risk rates without infinite-dimensional exploits
• Expands learnable measures via infinite-dimensional pathologies