A Fully Polynomial-Time Algorithm for Robustly Learning Halfspaces over the Hypercube

2025-11-10

AI Summary

This work studies robust learning of halfspaces over the Boolean hypercube under uniform distribution with adversarial contamination—where both samples and labels may be arbitrarily corrupted. Prior to this, even under label-only contamination in the agnostic setting, no fully polynomial-time algorithm was known. We present the first fully polynomial-time algorithm for learning halfspaces under adversarial contamination over discrete uniform distributions, breaking prior reliance on geometric properties of continuous distributions. Our approach builds upon a novel generalized linear model framework that requires only polylogarithmic dependence on the Lipschitz constant of the activation function, combined with noise-resilient constructions and robust statistical techniques. The algorithm achieves an error bound of η^O(1) + ε, where η is the contamination rate, matching optimal rates in both the agnostic and stronger contamination models—significantly improving upon existing super-polynomial-time methods.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We give the first fully polynomial-time algorithm for learning halfspaces with respect to the uniform distribution on the hypercube in the presence of contamination, where an adversary may corrupt some fraction of examples and labels arbitrarily. We achieve an error guarantee of $eta^{O(1)}+epsilon$ where $eta$ is the noise rate. Such a result was not known even in the agnostic setting, where only labels can be adversarially corrupted. All prior work over the last two decades has a superpolynomial dependence in $1/epsilon$ or succeeds only with respect to continuous marginals (such as log-concave densities). Previous analyses rely heavily on various structural properties of continuous distributions such as anti-concentration. Our approach avoids these requirements and makes use of a new algorithm for learning Generalized Linear Models (GLMs) with only a polylogarithmic dependence on the activation function's Lipschitz constant. More generally, our framework shows that supervised learning with respect to discrete distributions is not as difficult as previously thought.

Problem

• Learning halfspaces with adversarial contamination on hypercube
• Achieving polynomial-time error guarantee with noise rate
• Avoiding structural dependency on continuous distribution properties

Innovation

• Fully polynomial-time robust halfspace learning algorithm
• Achieves error guarantee with noise rate and epsilon
• Uses GLM learning with polylogarithmic Lipschitz dependence