Bagging Robustly Learns VC Classes with Linear Sample Complexity

2026-08-13

AI Summary

This work addresses the problem of learning adversarially robust predictors at test time, focusing on achieving robust learnability for VC classes under controlled sample complexity. The authors propose a novel approach that combines Bagging with Robust Empirical Risk Minimization (RERM): by training RERM models independently on O(d*) bootstrap samples and aggregating their predictions via majority voting. This method establishes, for the first time, that VC classes are robustly learnable with sample complexity linear in their VC dimension d*. Furthermore, under the RERM oracle model, the authors prove a matching lower bound of Ω(d*) oracle calls, thereby improving upon prior exponential upper bounds and demonstrating the theoretical optimality of their algorithm.

Abstract

We revisit the problem of learning predictors robust to adversarial examples at test-time. We prove that VC classes are adversarially robustly learnable with sample complexity linear in the VC dimension $d$, providing an exponential improvement over the previous upper bound of Montasser, Hanneke, and Srebro (2019). Remarkably, this result is achieved with a simple improper algorithm that combines the classic heuristic bagging (bootstrap aggregation) of Breiman (1996) with robust empirical risk minimization (RERM). Our algorithm computes RERMs on $O(d^\star)$ independent bootstrap samples and outputs their majority vote, where $d^\star$ denotes the dual VC dimension. We complement this result with a lower bound showing that this is unavoidable: in general, any learner in this oracle model requires $Ω(d^\star)$ calls to an RERM oracle, even when given arbitrarily many training examples.

Problem

• adversarial robustness
• VC classes
• sample complexity
• robust learning
• test-time robustness

Innovation

• adversarial robustness
• VC classes
• bagging
• sample complexity
• robust empirical risk minimization