Reliably detecting model failures in deployment without labels

2025-06-05

AI Summary

This paper addresses the reliable detection of post-deployment performance degradation (PDD) in unlabeled model-serving scenarios. We formally define the PDD monitoring task as distinguishing benign distributional shifts from genuine performance deterioration. To this end, we propose D3M—a label-free, gradient-free monitoring framework that leverages predictive disagreement across multiple models. We theoretically establish its low false-positive rate under non-degrading shifts and provide sample-complexity guarantees. By unifying theoretical analysis with empirical risk estimation, D3M achieves significant improvements over state-of-the-art baselines on standard benchmarks and a large-scale real-world internal medicine dataset. Our method delivers a verifiable, automated alerting mechanism for performance degradation in high-stakes machine learning systems.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

The distribution of data changes over time; models operating operating in dynamic environments need retraining. But knowing when to retrain, without access to labels, is an open challenge since some, but not all shifts degrade model performance. This paper formalizes and addresses the problem of post-deployment deterioration (PDD) monitoring. We propose D3M, a practical and efficient monitoring algorithm based on the disagreement of predictive models, achieving low false positive rates under non-deteriorating shifts and provides sample complexity bounds for high true positive rates under deteriorating shifts. Empirical results on both standard benchmark and a real-world large-scale internal medicine dataset demonstrate the effectiveness of the framework and highlight its viability as an alert mechanism for high-stakes machine learning pipelines.

Problem

• Detect model failures without labels in deployment
• Monitor post-deployment deterioration in dynamic environments
• Determine when to retrain models under data shifts

Innovation

• Monitors model disagreement for failure detection
• Achieves low false positives in non-deteriorating shifts
• Provides sample complexity bounds for true positives