Active Learning for Decision Trees with Provable Guarantees

2026-01-28

AI Summary

This work investigates the label query complexity of decision trees in active learning. Under the reasonable assumptions that path features are non-redundant and input data exhibit a regular grid structure, the study presents the first analysis of the disagreement coefficient for decision trees and introduces the first general-purpose active learning algorithm with a $(1+\varepsilon)$ multiplicative error guarantee. The proposed method reduces label complexity to polylogarithmic in the data size and establishes near-optimal dependence on $\varepsilon$, achieving performance guarantees that approach the theoretical lower bound.

Technology Category

• Machine Learning: Active Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper advances the theoretical understanding of active learning label complexity for decision trees as binary classifiers. We make two main contributions. First, we provide the first analysis of the disagreement coefficient for decision trees-a key parameter governing active learning label complexity. Our analysis holds under two natural assumptions required for achieving polylogarithmic label complexity, (i) each root-to-leaf path queries distinct feature dimensions, and (ii) the input data has a regular, grid-like structure. We show these assumptions are essential, as relaxing them leads to polynomial label complexity. Second, we present the first general active learning algorithm for binary classification that achieves a multiplicative error guarantee, producing a $(1+\epsilon)$-approximate classifier. By combining these results, we design an active learning algorithm for decision trees that uses only a polylogarithmic number of label queries in the dataset size, under the stated assumptions. Finally, we establish a label complexity lower bound, showing our algorithm's dependence on the error tolerance $\epsilon$ is close to optimal.

Problem

• active learning
• decision trees
• label complexity
• disagreement coefficient
• binary classification

Innovation

• active learning
• decision trees
• label complexity
• disagreement coefficient
• multiplicative error guarantee