Uniform Computability of PAC Learning

2026-01-26

AI Summary

This study investigates the computability and complexity of Probably Approximately Correct (PAC) learning under various forms of information—positive examples, negative examples, and complete data—within the framework of Weihrauch reducibility. It pioneers the systematic application of computability theory to classify PAC learning scenarios, establishing that proper learning from positive examples is Weihrauch-equivalent to the limit operation on Baire space, while improper learning corresponds to Weak Kőnig’s Lemma. The computational complexity of determining the VC dimension is shown to be equivalent to binary sorting under positive or full information, and to the Turing jump of the sorting problem under negative information. By integrating tools from the Borel hierarchy, diagonally non-computable (DNC) functions, and reductions from sorting problems, the paper precisely characterizes the Weihrauch degrees and Borel complexities of multiple PAC learning settings.

Technology Category

• Machine Learning: Learning Theory
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

We study uniform computability properties of PAC learning using Weihrauch complexity. We focus on closed concept classes, which are either represented by positive, by negative or by full information. Among other results, we prove that proper PAC learning from positive information is equivalent to the limit operation on Baire space, whereas improper PAC learning from positive information is closely related to Weak K\H{o}nig's Lemma and even equivalent to it, when we have some negative information about the admissible hypotheses. If arbitrary hypotheses are allowed, then improper PAC learning from positive information is still in a finitary DNC range, which implies that it is non-deterministically computable, but does not allow for probabilistic algorithms. These results can also be seen as a classification of the degree of constructivity of the Fundamental Theorem of Statistical Learning. All the aforementioned results hold if an upper bound of the VC dimension is provided as an additional input information. We also study the question of how these results are affected if the VC dimension is not given, but only promised to be finite or if concept classes are represented by negative or full information. Finally, we also classify the complexity of the VC dimension operation itself, which is a problem that is of independent interest. For positive or full information it turns out to be equivalent to the binary sorting problem, for negative information it is equivalent to the jump of sorting. This classification allows also conclusions regarding the Borel complexity of PAC learnability.

Problem

• PAC learning
• computability
• VC dimension
• Weihrauch complexity
• concept classes

Innovation

• Weihrauch reducibility
• PAC learning
• VC dimension
• computable analysis
• constructive learning theory