Drift Localization using Conformal Predictions

2026-02-23

AI Summary

This work proposes a novel concept drift localization method based on conformal prediction to address the challenge of accurately identifying affected samples in high-dimensional, low-signal scenarios where traditional local detection approaches often fail. By constructing a non-parametric, distribution-free confidence mechanism, the method effectively pinpoints drift-affected instances, substantially enhancing the robustness and accuracy of monitoring systems. Notably, this is the first study to systematically integrate conformal prediction into concept drift localization, deliberately moving away from conventional local testing strategies. Extensive evaluations on multiple state-of-the-art image datasets demonstrate the method’s superior performance, achieving significant improvements over existing approaches in both localization accuracy and stability.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Local Search

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Concept drift -- the change of the distribution over time -- poses significant challenges for learning systems and is of central interest for monitoring. Understanding drift is thus paramount, and drift localization -- determining which samples are affected by the drift -- is essential. While several approaches exist, most rely on local testing schemes, which tend to fail in high-dimensional, low-signal settings. In this work, we consider a fundamentally different approach based on conformal predictions. We discuss and show the shortcomings of common approaches and demonstrate the performance of our approach on state-of-the-art image datasets.

Problem

• concept drift
• drift localization
• conformal predictions
• high-dimensional data
• distribution shift

Innovation

• concept drift
• drift localization
• conformal predictions
• high-dimensional data
• distribution shift