The Optimal Sample Complexity of Multiclass and List Learning

2026-04-27

AI Summary

This work resolves a long-standing open problem concerning the sample complexity of multiclass classification and list learning in terms of the Daniely–Shalev-Shwartz (DS) dimension. Addressing the previously unresolved √DS gap between upper and lower bounds, the authors leverage an algebraic characterization of multiclass hypothesis classes due to Hanneke et al., combined with hypergraph density analysis and an extension of VC theory, to prove that the maximum hypergraph density of any multiclass hypothesis class is bounded above by its DS dimension. This result establishes, for the first time, the optimal dependence of sample complexity on the DS dimension, fully closing the theoretical gap and confirming the conjecture posed by Daniely and Shalev-Shwartz (2014).

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Learning & Optimization for CV
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Machine learning and data science for the Web

Abstract

While the optimal sample complexity of binary classification in terms of the VC dimension is well-established, determining the optimal sample complexity of multiclass classification has remained open. The appropriate complexity parameter for multiclass classification is the DS dimension, and despite significant efforts, a gap of $\sqrt{\text{DS}}$ has persisted between the upper and lower bounds on sample complexity. Recent work by Hanneke et al. (2026) shows a novel algebraic characterization of multiclass hypothesis classes in terms of their DS dimension. Building up on this, we show that the maximum hypergraph density of any multiclass hypothesis class is upper-bounded by its DS dimension. This proves a longstanding conjecture of Daniely and Shalev-Shwartz (2014). As a consequence, we determine the optimal dependence of the sample complexity on the DS dimension for multiclass as well as list learning.

Problem

• multiclass classification
• sample complexity
• DS dimension
• list learning
• VC dimension

Innovation

• multiclass classification
• sample complexity
• DS dimension
• hypergraph density
• list learning