covariate shift detection

Existing methods for detecting simultaneous changes in the mean vector and covariance matrix of high-dimensional data suffer from reduced detection power and inaccurate change-point localization due to separate modeling of these two types of changes. Method: This paper proposes a unified changepoint detection and localization framework. Its core innovation is the first theoretical demonstration of asymptotic independence between test statistics for mean and covariance changes, enabling p-value fusion via Fisher’s method to construct an adaptive changepoint estimator. Leveraging high-dimensional statistical inference, we design a decouplable joint test statistic and integrate asymptotic distribution theory with p-value combination to achieve integrated detection and precise localization. Results: Theoretical analysis and extensive simulations demonstrate that our method significantly improves detection power and localization accuracy under simultaneous changes—particularly excelling in high-dimensional sparse changepoint settings.

This paper addresses the unreliability of causal and predictive parameter estimation under covariate shift. We propose a fully automated debiasing machine learning framework that eliminates regularization bias solely through parameter definition—without requiring explicit bias modeling. Our approach innovatively integrates training and target data within a unified debiasing mechanism, combining data fusion, high-dimensional statistical inference, doubly robust estimation, and the difference-in-differences (DID) principle—all under an unconfoundedness assumption. We establish theoretical guarantees of consistency and asymptotic normality. In simulation studies and an empirical analysis of minimum wage effects on teenage employment, our method reduces estimation bias by over 40% on average compared to benchmark approaches, while substantially improving estimation accuracy and robustness.

In AI deployment for medical imaging, data distribution shifts frequently cause abrupt performance degradation and increased misdiagnosis risk. Existing methods can only detect the presence of shift but fail to identify its specific type—e.g., covariate shift, prior (concept) shift, or compound shift—hindering root-cause analysis and targeted mitigation. This paper proposes the first unsupervised framework for data shift type identification. It introduces a novel joint shift detection mechanism that synergistically leverages self-supervised encoder representations and task-model outputs. By integrating feature distribution comparison, unsupervised clustering, and multimodal image modeling, the method achieves high-accuracy shift-type discrimination across three major imaging modalities—chest X-ray, mammography, and fundus photography—and five realistic shift scenarios. Evaluated on four large public medical imaging datasets, it significantly enhances the robustness and interpretability of clinical AI systems.

To address the challenge of unsupervised model performance estimation under covariate shift—where ground-truth labels are unavailable or delayed post-deployment—this paper proposes the Probability-Adaptive Performance Estimation (PAPE) framework. PAPE requires neither access to true labels nor knowledge of the original model’s architecture or feature representations; it operates solely on the model’s probabilistic outputs and confidence scores. By jointly leveraging density ratio estimation and performance generalization bound theory, PAPE models prediction distributions and applies adaptive reweighting to yield unbiased estimates of arbitrary classification metrics—without assuming a specific shift form or resorting to feature learning or generative modeling. Extensive evaluation across 900+ real-world census dataset–model combinations demonstrates that PAPE reduces mean absolute error by 37% compared to state-of-the-art proxy metrics and drift detection methods, significantly enhancing the reliability and generality of model monitoring in production environments.

This work addresses the lack of theoretical guarantees on training-conditional coverage—i.e., coverage under the training data distribution—of conformal prediction under covariate shift. We first systematically investigate its upper-bound characterization and controllability. We derive a weighted Dvoretzky–Kiefer–Wolfowitz inequality to establish tight, provable training-conditional coverage bounds for split conformal prediction under nearly assumption-free conditions. Furthermore, leveraging algorithmic uniform stability, we provide the first training-conditional coverage guarantees for full conformal and jackknife+ methods. Our results demonstrate that all three mainstream conformal prediction frameworks achieve controllable training-conditional coverage under covariate shift, with split conformal yielding bounds that are both minimally assumption-dependent and tight. This work fills a critical theoretical gap in conditional coverage analysis of conformal prediction beyond the i.i.d. setting.

This work addresses the challenge of predicting performance changes when a source-domain model is replaced by a new one. To this end, the authors propose TRACE, a novel framework that, for the first time, decomposes the risk difference between two models under covariate shift into four interpretable components: two generalization gaps, a model change penalty, and a covariate shift penalty. The framework establishes a computable upper bound to diagnose the causes of performance degradation. TRACE estimates model sensitivity via high-quantile input gradients, quantifies data distribution shift using either optimal transport (OT) or maximum mean discrepancy (MMD), and measures model change through output distances on target samples. Experiments demonstrate that TRACE’s diagnostic scores exhibit strong monotonic correlation with actual performance degradation and achieve superior performance in deployment gating, as measured by AUROC and AUPRC, thereby enabling label-efficient and safe model replacement.

When predictive models are deployed in new environments, their performance often degrades due to covariate shift and selective labeling, which jointly obscure accurate assessment of the true target risk. This work proposes an unbiased risk estimation method that integrates double machine learning with influence functions to simultaneously address both sources of bias for the first time. The approach is model-agnostic and compatible with general loss functions, constructing a corrected target risk estimator via nonparametric and plug-in techniques. Experiments on eICU electronic health record data demonstrate that the proposed method significantly outperforms baselines that handle only one type of bias or naively combine existing approaches, yielding more accurate tracking of the true target risk.

This work addresses the challenge of verifying covariate balance in covariate shift adaptation by proposing a sequentially valid, anytime-stoppable validation framework. Built upon time-uniform confidence sequences, the method dynamically monitors covariate balance for a pre-specified function class within a prescribed tolerance band and terminates as soon as all target moments fall within this band, thereby certifying balance. Its key contribution lies in providing, for the first time, a locally and absolutely valid certification of balance for any adjustment strategy, supporting data-dependent stopping times while rigorously controlling the probability of erroneous balance confirmation. Integrated with KL-divergence-based drift diagnostics and detection of admissible adjustment regions, experiments demonstrate the method’s superior performance in error rate control, locality with respect to function classes, effectiveness in drift diagnosis, and guaranteed coverage in conformal prediction.

This work addresses the challenge of accurately attributing detected change points in multivariate time series to specific subsets of variables. The authors propose a post-hoc, nonparametric testing framework that, after an offline change point has been identified, determines whether the change occurs in one of two pre-specified coordinate blocks or in both. Built upon two-sample nonparametric hypothesis testing, the method offers rigorous theoretical guarantees for Type I error control. Empirical evaluations on both synthetic and real-world datasets demonstrate that the proposed approach achieves high attribution accuracy and strong robustness in identifying the components responsible for the change.

This work addresses the challenge of mixed distributional shift, which simultaneously involves systematic bias and random perturbations—a setting where conventional reweighting methods struggle to balance bias correction with uncertainty quantification. The authors propose a novel framework that explicitly disentangles these two sources of shift: systematic bias is corrected via Augmented Inverse Distance Weighting (AIDW) and Augmented Inverse Hybrid Weighting (AIHW), while residual random perturbations are modeled as distributional uncertainty. Robust inference is achieved through variance-optimal data pooling. The method incorporates distributional distance to govern the bias–variance trade-off, offering both asymptotic theoretical guarantees and practical guidance for hyperparameter tuning. Evaluated on three real-world multicenter datasets, the approach substantially reduces mean squared error and improves empirical coverage, demonstrating particular robustness in scenarios where covariate shift correction typically underestimates uncertainty.