Guided Random Forest and its application to data approximation

2019-09-02

arXiv.org

AI Summary

To bridge the theoretical gap between decision trees and boosting methods—and address their high generalization error—this paper proposes Guided Random Forests (GRAF). GRAF is the first random forest variant to incorporate a global partitioning mechanism, synergizing oblique decision trees to achieve both locally optimal splits and globally optimized tree structures. It introduces a “local splitting → global ensemble” strategy, unifying support for classification and data approximation tasks. Methodologically, GRAF reconciles the bias–variance trade-off discrepancy between single-tree models and boosting algorithms, while establishing the first random-forest-based paradigm for data approximation. Extensive experiments across 115 benchmark datasets demonstrate that GRAF achieves statistically significant improvements—or competitive performance—over state-of-the-art ensemble methods (e.g., XGBoost, LightGBM, standard RF) in both classification accuracy and generalization capability. Crucially, GRAF’s theoretical generalization error bound is strictly lower than prior forest models, with empirical validation confirming this reduction.

Abstract

We present a new way of constructing an ensemble classifier, named the Guided Random Forest (GRAF) in the sequel. GRAF extends the idea of building oblique decision trees with localized partitioning to obtain a global partitioning. We show that global partitioning bridges the gap between decision trees and boosting algorithms. We empirically demonstrate that global partitioning reduces the generalization error bound. Results on 115 benchmark datasets show that GRAF yields comparable or better results on a majority of datasets. We also present a new way of approximating the datasets in the framework of random forests.

Problem

• Develops Guided Random Forest for ensemble classification
• Bridges gap between decision trees and boosting
• Improves generalization error via global partitioning

Innovation

• GRAF extends oblique decision trees globally
• Global partitioning bridges trees and boosting
• GRAF approximates datasets within random forests