IncA-DES: An incremental and adaptive dynamic ensemble selection approach using online K-d tree neighborhood search for data streams with concept drift

2025-07-16

AI Summary

To address performance degradation of traditional classifiers under concept drift in data streams, this paper proposes an incremental adaptive dynamic ensemble selection method. The approach comprises three key contributions: (1) an online K-d tree supporting efficient insertion/deletion and inherent handling of class imbalance, drastically accelerating k-nearest neighbor search; (2) a local-overlap-based classification filtering mechanism to suppress redundant expert invocation; and (3) a lightweight concept drift detector tightly coupled with the ensemble to enable adaptive reconfiguration of the local expert set. Evaluated across multi-label availability settings—including fully labeled, partially labeled, and unlabeled scenarios—the method consistently outperforms mainstream baselines in accuracy while achieving substantial inference speedup with negligible accuracy loss. The implementation is publicly available.

Technology Category

• Machine Learning: Ensemble Methods
• Data Mining & Knowledge Management: Data Stream Mining
• Search and Optimization: Local Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Data streams pose challenges not usually encountered in batch-based ML. One of them is concept drift, which is characterized by the change in data distribution over time. Among many approaches explored in literature, the fusion of classifiers has been showing good results and is getting growing attention. DS methods, due to the ensemble being instance-based, seem to be an efficient choice under drifting scenarios. However, some attention must be paid to adapting such methods for concept drift. The training must be done in order to create local experts, and the commonly used neighborhood-search DS may become prohibitive with the continuous arrival of data. In this work, we propose IncA-DES, which employs a training strategy that promotes the generation of local experts with the assumption that different regions of the feature space become available with time. Additionally, the fusion of a concept drift detector supports the maintenance of information and adaptation to a new concept. An overlap-based classification filter is also employed in order to avoid using the DS method when there is a consensus in the neighborhood, a strategy that we argue every DS method should employ, as it was shown to make them more applicable and quicker. Moreover, aiming to reduce the processing time of the kNN, we propose an Online K-d tree algorithm, which can quickly remove instances without becoming inconsistent and deals with unbalancing concerns that may occur in data streams. Experimental results showed that the proposed framework got the best average accuracy compared to seven state-of-the-art methods considering different levels of label availability and presented the smaller processing time between the most accurate methods. Additionally, the fusion with the Online K-d tree has improved processing time with a negligible loss in accuracy. We have made our framework available in an online repository.

Problem

• Handles concept drift in data streams adaptively
• Improves efficiency of dynamic ensemble selection
• Reduces processing time with Online K-d tree

Innovation

• Incremental dynamic ensemble selection for data streams
• Online K-d tree for efficient neighborhood search
• Concept drift detector for adaptive classification