Leveraging Predictive Equivalence in Decision Trees

2025-06-17

AI Summary

Decision trees, while interpretable, suffer from prediction equivalence: structurally distinct trees can represent identical decision boundaries, leading to inconsistent feature importance estimates, unreliable behavior under test-time feature missingness, and arbitrary selection among equivalent models by existing optimization methods. This work introduces the first boundary-faithful, equivalence-free Boolean logic representation of decision trees—eliminating structural redundancy at its root. Building upon this representation, we provide the first rigorous characterization of decision-tree robustness to feature missingness during inference, redefine feature importance to ensure consistency and cross-model comparability, and design a prediction-cost-optimal Boolean inference algorithm. Experiments demonstrate that our approach significantly improves predictive stability under missing features, unifies interpretability evaluation across models, and achieves both theoretical rigor and computational efficiency.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Description Logics

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Decision trees are widely used for interpretable machine learning due to their clearly structured reasoning process. However, this structure belies a challenge we refer to as predictive equivalence: a given tree's decision boundary can be represented by many different decision trees. The presence of models with identical decision boundaries but different evaluation processes makes model selection challenging. The models will have different variable importance and behave differently in the presence of missing values, but most optimization procedures will arbitrarily choose one such model to return. We present a boolean logical representation of decision trees that does not exhibit predictive equivalence and is faithful to the underlying decision boundary. We apply our representation to several downstream machine learning tasks. Using our representation, we show that decision trees are surprisingly robust to test-time missingness of feature values; we address predictive equivalence's impact on quantifying variable importance; and we present an algorithm to optimize the cost of reaching predictions.

Problem

• Addresses predictive equivalence in decision trees
• Improves robustness to missing feature values
• Optimizes variable importance quantification accuracy

Innovation

• Boolean logical representation for decision trees
• Robustness to test-time missing feature values
• Algorithm optimizes prediction cost