Necessary and Sufficient Conditions for Optimal Decision Trees using Dynamic Programming

Unknown Date

AI Summary

Decision tree global optimization under big data faces a fundamental trade-off among accuracy, model size, and interpretability, exacerbated by the intractability of existing methods for arbitrary objective functions. Method: This paper establishes, for the first time, necessary and sufficient conditions for dynamic programming (DP) solvability of decision trees, thereby lifting restrictive assumptions on objective function forms. We propose a general DP framework supporting arbitrary separable objectives and constraints, integrating tree-structure decomposition with separability analysis. Contribution/Results: Experiments across five diverse application domains demonstrate that our method guarantees global optimality and preserves model interpretability while achieving substantial scalability improvements—outperforming generic optimization solvers by a wide margin in both efficiency and solution quality.

Abstract

Global optimization of decision trees has shown to be promising in terms of accuracy, size, and consequently human comprehensibility. However, many of the methods used rely on general-purpose solvers for which scalability remains an issue. Dynamic programming methods have been shown to scale much better because they exploit the tree structure by solving subtrees as independent subproblems. However, this only works when an objective can be optimized separately for subtrees. We explore this relationship in detail and show the necessary and sufficient conditions for such separability and generalize previous dynamic programming approaches into a framework that can optimize any combination of separable objectives and constraints. Experiments on five application domains show the general applicability of this framework, while outperforming the scalability of general-purpose solvers by a large margin.

Problem

• Big Data
• Decision Trees
• Optimization

Innovation

• Dynamic Programming
• Optimal Decision Tree
• Big Data Analysis