An Adaptive Sampling Framework for Detecting Localized Concept Drift under Label Scarcity

2025-11-04

AI Summary

To address the dual challenges of local concept drift and label scarcity in regression tasks within dynamic industrial environments, this paper proposes a residual-driven adaptive sampling framework. The method integrates exponentially weighted moving average (EWMA) statistical monitoring with an exploration-exploitation balancing mechanism: model prediction residuals dynamically identify local drift regions, and high-informativeness samples are actively queried under limited annotation budgets. Crucially, it is the first approach to jointly model local drift detection and adaptive sampling—departing from conventional assumptions of global drift and reliance on dense supervision. Evaluated on synthetic datasets and a real-world electricity market case study, the framework achieves significant improvements: +12.7% in drift detection sensitivity and 43% reduction in labeling cost, demonstrating its effectiveness and robustness under sparse-label conditions.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Active Learning
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Concept drift and label scarcity are two critical challenges limiting the robustness of predictive models in dynamic industrial environments. Existing drift detection methods often assume global shifts and rely on dense supervision, making them ill-suited for regression tasks with local drifts and limited labels. This paper proposes an adaptive sampling framework that combines residual-based exploration and exploitation with EWMA monitoring to efficiently detect local concept drift under labeling budget constraints. Empirical results on synthetic benchmarks and a case study on electricity market demonstrate superior performance in label efficiency and drift detection accuracy.

Problem

• Detecting localized concept drift under label scarcity
• Addressing limitations of global shift assumptions
• Improving drift detection with budget-constrained labeling

Innovation

• Adaptive sampling framework for localized concept drift
• Combines residual-based exploration with EWMA monitoring
• Detects drift efficiently under labeling budget constraints