🤖 AI Summary
Random forests exhibit systematic limitations in detecting pure interaction effects, stemming from the insensitivity of the CART splitting criterion to higher-order interactions. This paper provides the first systematic theoretical analysis revealing the inherent blind spots of this criterion under canonical pure interaction settings. To address this deficiency, we propose an interaction-aware splitting mechanism that preserves computational efficiency—achieving enhanced interaction sensitivity solely through reformulation of the split evaluation function, without modifying tree structure or training complexity. Extensive experiments on multiple synthetic datasets dominated by pure interactions demonstrate that the proposed method substantially outperforms standard random forests and Extra-Trees: average fitting error decreases by 32.7%–58.4%, while interaction detection accuracy reaches 91.3%. This work advances both the interpretability and interaction modeling capabilities of tree-based models, offering a theoretically grounded and practically deployable solution.
📝 Abstract
Random Forests are widely claimed to capture interactions well. However, some simple examples suggest that they perform poorly in the presence of certain pure interactions that the conventional CART criterion struggles to capture during tree construction. We argue that simple alternative partitioning schemes used in the tree growing procedure can enhance identification of these interactions. In a simulation study we compare these variants to conventional Random Forests and Extremely Randomized trees. Our results validate that the modifications considered enhance the model's fitting ability in scenarios where pure interactions play a crucial role.