Enriched Functional Tree-Based Classifiers: A Novel Approach Leveraging Derivatives and Geometric Features

2024-09-26

arXiv.org

AI Summary

This paper addresses supervised classification of high-dimensional time series via scalar-valued functional data. We propose a novel framework integrating functional data analysis with tree-based ensemble learning. Methodologically, we introduce functional preprocessing, derivative estimation, and extraction of differential geometric features—including curvature and torsion—systematically embedded for the first time into functional decision trees (FCT), functional random forests (FRF), and functional XGBoost/LightGBM. These geometric features substantially enhance discriminative power and robustness. Extensive experiments across seven real-world datasets and six synthetic benchmarks demonstrate that our approach achieves an average accuracy gain of 5.2% over conventional functional classifiers and reduces prediction variance by 37%, confirming its effectiveness and superior generalization capability.

Technology Category

• Machine Learning: Ensemble Methods
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Machine learning and data science for the Web

Abstract

The positioning of this research falls within the scalar-on-function classification literature, a field of significant interest across various domains, particularly in statistics, mathematics, and computer science. This study introduces an advanced methodology for supervised classification by integrating Functional Data Analysis (FDA) with tree-based ensemble techniques for classifying high-dimensional time series. The proposed framework, Enriched Functional Tree-Based Classifiers (EFTCs), leverages derivative and geometric features, benefiting from the diversity inherent in ensemble methods to further enhance predictive performance and reduce variance. While our approach has been tested on the enrichment of Functional Classification Trees (FCTs), Functional K-NN (FKNN), Functional Random Forest (FRF), Functional XGBoost (FXGB), and Functional LightGBM (FLGBM), it could be extended to other tree-based and non-tree-based classifiers, with appropriate considerations emerging from this investigation. Through extensive experimental evaluations on seven real-world datasets and six simulated scenarios, this proposal demonstrates fascinating improvements over traditional approaches, providing new insights into the application of FDA in complex, high-dimensional learning problems.

Problem

• Enhance supervised classification of high-dimensional time series
• Integrate Functional Data Analysis with tree-based ensemble methods
• Improve predictive performance using derivative and geometric features

Innovation

• Functional Data Analysis integration
• Derivative and geometric features
• Tree-based ensemble methods