Higher-arity PAC learning, VC dimension and packing lemma

2025-10-02

AI Summary

Classical VC theory and PAC learning lack a rigorous foundation for high-dimensional learning scenarios involving structured, multi-way dependencies. Method: This work generalizes VC theory and PAC learning to *n*-fold product spaces under product measures, introducing the notions of higher-order VCₙ-dimension and PACₙ learnability. Leveraging tools from model theory, combinatorics, and probabilistic methods, it establishes the first formal definition of VCₙ-dimension, extends Haussler’s packing lemma, and introduces a *shattered hypergraph regularity lemma*. Contribution/Results: It provides a complete characterization of PACₙ learnability—namely, a hypothesis class is PACₙ-learnable if and only if its VCₙ-dimension is finite. This unifies and recovers key results from several recent works in higher-order statistical learning, thereby establishing the first systematic theoretical framework for learning over *n*-ary relations and high-order interactions.

Technology Category

• Machine Learning: Learning Theory
• Game Theory and Economic Paradigms: Adversarial Learning
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The aim of this note is to overview some of our work in Chernikov, Towsner'20 (arXiv:2010.00726) developing higher arity VC theory (VC$_n$ dimension), including a generalization of Haussler packing lemma, and an associated tame (slice-wise) hypergraph regularity lemma; and to demonstrate that it characterizes higher arity PAC learning (PAC$_n$ learning) in $n$-fold product spaces with respect to product measures introduced by Kobayashi, Kuriyama and Takeuchi'15. We also point out how some of the recent results in arXiv:2402.14294, arXiv:2505.15688, arXiv:2509.20404 follow from our work in arXiv:2010.00726.

Problem

• Developing higher-arity VC theory and VC$_n$ dimension
• Generalizing Haussler packing lemma for hypergraph regularity
• Characterizing higher-arity PAC learning in product spaces

Innovation

• Developed higher arity VC theory with VC$_n$ dimension
• Generalized Haussler packing lemma for hypergraphs
• Introduced tame hypergraph regularity lemma for PAC$_n$ learning