Hardness of High-Dimensional Linear Classification

2026-03-19

AI Summary

This work addresses the long-standing theoretical gap in the computational complexity of high-dimensional linear classification—formally, the maximum-margin halfspace problem—by establishing tight conditional and unconditional lower bounds. Under fine-grained complexity assumptions, including affine degeneracy detection and the k-SUM conjecture, the paper proves conditional time lower bounds of ~Ω(n^d) and ~Ω(n^{d/2}), respectively. Moreover, in an unconditional query model restricted to label-only (lateral) access, it establishes a matching ~Ω(n^d) lower bound. By integrating techniques from computational geometry and problem reductions, these results align precisely with the best-known upper bounds of ~O(n^d) and ~O(1/ε^d), thereby characterizing the intrinsic hardness of the problem across multiple computational models and closing a fundamental gap in the theoretical understanding of high-dimensional linear classification.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Large-scale security measurements

Abstract

We establish new exponential in dimension lower bounds for the Maximum Halfspace Discrepancy problem, which models linear classification. Both are fundamental problems in computational geometry and machine learning in their exact and approximate forms. However, only $O(n^d)$ and respectively $\tilde O(1/\varepsilon^d)$ upper bounds are known and complemented by polynomial lower bounds that do not support the exponential in dimension dependence. We close this gap up to polylogarithmic terms by reduction from widely-believed hardness conjectures for Affine Degeneracy testing and $k$-Sum problems. Our reductions yield matching lower bounds of $\tildeΩ(n^d)$ and respectively $\tildeΩ(1/\varepsilon^d)$ based on Affine Degeneracy testing, and $\tildeΩ(n^{d/2})$ and respectively $\tildeΩ(1/\varepsilon^{d/2})$ conditioned on $k$-Sum. The first bound also holds unconditionally if the computational model is restricted to make sidedness queries, which corresponds to a widely spread setting implemented and optimized in many contemporary algorithms and computing paradigms.

Problem

• High-Dimensional Linear Classification
• Maximum Halfspace Discrepancy
• Computational Complexity
• Exponential Lower Bounds
• Computational Geometry

Innovation

• Maximum Halfspace Discrepancy
• computational hardness
• high-dimensional classification
• Affine Degeneracy
• k-Sum