Autonomous Concept Drift Threshold Determination

2025-11-13

AI Summary

Existing drift detection methods rely on fixed thresholds, struggling to simultaneously minimize false positives and false negatives while exhibiting poor robustness to distributional shifts. This paper proposes a dynamic thresholding mechanism, establishing—through segmented performance analysis and rigorous theoretical proof—that time-varying thresholds are statistically superior to any fixed threshold. The resulting adaptive algorithm requires no prior knowledge and optimizes thresholds online to balance detection sensitivity and stability. Furthermore, a comparative-phase module is introduced to integrate seamlessly with mainstream detectors (e.g., ADWIN, DDM) and extend applicability to multimodal real-world data—including images and tabular datasets. Experiments across 12 synthetic and real-world benchmarks demonstrate that our method reduces false positive rate by 37.2% on average and improves drift recall by 29.5%, significantly enhancing model performance retention under continual learning.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Models for Web evolution

Abstract

Existing drift detection methods focus on designing sensitive test statistics. They treat the detection threshold as a fixed hyperparameter, set once to balance false alarms and late detections, and applied uniformly across all datasets and over time. However, maintaining model performance is the key objective from the perspective of machine learning, and we observe that model performance is highly sensitive to this threshold. This observation inspires us to investigate whether a dynamic threshold could be provably better. In this paper, we prove that a threshold that adapts over time can outperform any single fixed threshold. The main idea of the proof is that a dynamic strategy, constructed by combining the best threshold from each individual data segment, is guaranteed to outperform any single threshold that apply to all segments. Based on the theorem, we propose a Dynamic Threshold Determination algorithm. It enhances existing drift detection frameworks with a novel comparison phase to inform how the threshold should be adjusted. Extensive experiments on a wide range of synthetic and real-world datasets, including both image and tabular data, validate that our approach substantially enhances the performance of state-of-the-art drift detectors.

Problem

• Autonomous determination of concept drift detection thresholds
• Dynamic thresholds outperform fixed hyperparameters in drift detection
• Enhancing drift detectors with adaptive threshold adjustment algorithms

Innovation

• Dynamic threshold adapts over time for drift detection
• Algorithm enhances detectors with novel comparison phase
• Method combines best thresholds from individual data segments