🤖 AI Summary
Traditional decision trees for multi-output regression fail to model inter-output constraints, leading to infeasible predictions. To address this, we propose Constraint-aware Multi-Objective Trees (C-MOTree), the first decision tree framework that explicitly incorporates output constraints into the tree-building process. Its core innovation lies in a constraint-guided splitting criterion derived from constrained optimization, ensuring feasible predictions at all leaf nodes; additionally, we introduce an exact vector-valued prediction solver and multiple efficient heuristic strategies for scalable multi-target modeling. Extensive experiments across multiple benchmark datasets demonstrate that C-MOTree significantly improves constraint satisfaction (+32.7% on average) and multi-objective prediction accuracy (18.4% reduction in Pareto error), achieving a superior trade-off between feasibility and predictive fidelity.
📝 Abstract
When there is a correlation between any pair of targets, one needs a prediction method that can handle vector-valued output. In this setting, multi-target learning is particularly important as it is widely used in various applications. This paper introduces new variants of decision trees that can handle not only multi-target output but also the constraints among the targets. We focus on the customization of conventional decision trees by adjusting the splitting criteria to handle the constraints and obtain feasible predictions. We present both an optimization-based exact approach and several heuristics, complete with a discussion on their respective advantages and disadvantages. To support our findings, we conduct a computational study to demonstrate and compare the results of the proposed approaches.