Centroid Decision Forest

2025-03-25

AI Summary

To address the weak modeling capability and poor interpretability of conventional decision trees in high-dimensional classification, this paper proposes a class-center-driven ensemble learning framework. Methodologically, it introduces (1) a Class Separability Score (CSS) to guide feature selection; (2) class centroids—computed as the mean vectors of samples per class within learned subspaces—as splitting criteria, replacing scalar thresholds and enabling Euclidean distance–based, class-center-oriented node partitioning; and (3) a lightweight ensemble forest architecture. The approach jointly enhances structural modeling capacity and decision interpretability. Evaluated on 23 high-dimensional benchmark datasets, it consistently outperforms state-of-the-art classifiers, achieving superior classification accuracy and Cohen’s kappa scores. These results validate its effectiveness, robustness, and practical utility in high-dimensional settings.

Technology Category

• Machine Learning: Ensemble Methods
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

This paper introduces the centroid decision forest (CDF), a novel ensemble learning framework that redefines the splitting strategy and tree building in the ordinary decision trees for high-dimensional classification. The splitting approach in CDF differs from the traditional decision trees in theat the class separability score (CSS) determines the selection of the most discriminative features at each node to construct centroids of the partitions (daughter nodes). The splitting criterion uses the Euclidean distance measurements from each class centroid to achieve a splitting mechanism that is more flexible and robust. Centroids are constructed by computing the mean feature values of the selected features for each class, ensuring a class-representative division of the feature space. This centroid-driven approach enables CDF to capture complex class structures while maintaining interpretability and scalability. To evaluate CDF, 23 high-dimensional datasets are used to assess its performance against different state-of-the-art classifiers through classification accuracy and Cohen's kappa statistic. The experimental results show that CDF outperforms the conventional methods establishing its effectiveness and flexibility for high-dimensional classification problems.

Problem

• Novel ensemble learning for high-dimensional classification
• Class separability score guides discriminative feature selection
• Centroid-driven splitting enhances flexibility and robustness

Innovation

• Centroid-driven splitting with class separability score
• Euclidean distance-based flexible splitting mechanism
• Mean feature values for class-representative centroids