Decision Tree Learning on Product Spaces

2026-05-13

AI Summary

This work addresses the lack of theoretical guarantees for greedy decision tree construction under non-uniform data distributions by extending existing analyses from uniform to arbitrary product distributions. Leveraging tools from Boolean function analysis—specifically influence-based arguments—and jointly characterizing average and maximum tree depth through a recursive splitting strategy, the authors establish a precise relationship between the approximation accuracy of greedy algorithms and the resulting tree size. In the full binary tree setting, the derived bound improves upon prior results and leads to a practical algorithm that requires no prior knowledge of distributional parameters. Theoretically, they prove that a greedy algorithm can construct an ε-approximate decision tree whose size is at most exp(Δ_opt·D_opt·log(e/ε)), yielding a significant improvement over previous bounds under certain conditions.

Abstract

Decision tree learning has long been a central topic in theoretical computer science, driven by its practical importance. A fundamental and widely used method for decision tree construction is the top-down greedy heuristic, which recursively splits on the most influential variable. Despite its empirical success, theoretical analysis of this heuristic has been limited. A recent breakthrough by Blanc et al. (ITCS, 2020) provided the first rigorous theoretical guarantees for the greedy approach, but only under the uniform distribution. We extend this analysis to the more general and practically relevant setting of arbitrary product distributions. Our main result shows that for any function $f$ computable by an optimal decision tree of size $s$, maximum depth $D_{\text{opt}}$, and average depth $Δ_{\text{opt}}$, the greedy heuristic constructs an $ε$-approximating tree whose size grows at most with $\exp\bigl(Δ_{\text{opt}} D_{\text{opt}} \log(e/ε)\bigr)$. In the special case where the optimal tree is a full binary tree, this bound improves upon the bound of Blanc et al. and holds under a strictly broader class of distributions. Moreover, we present an algorithm based on the top-down greedy heuristic that is entirely parameter-free -- it requires no prior knowledge of the optimal tree's size or depth -- offering a practical advantage over Blanc et al.'s method.

Problem

• Decision Tree Learning
• Greedy Heuristic
• Product Distributions
• Theoretical Guarantees
• Approximation Bounds

Innovation

• decision tree learning
• greedy heuristic
• product distributions
• theoretical guarantees
• parameter-free algorithm