Prediction-Powered Risk Monitoring of Deployed Models for Detecting Harmful Distribution Shifts

2026-02-02

AI Summary

This work addresses the challenge of timely detecting harmful distribution shifts in deployed models under dynamic environments where labeled data are scarce. To this end, the authors propose Predictive-Driven Risk Monitoring (PPRM), a novel approach that introduces predictive-driven inference into risk monitoring by integrating synthetic labels with a small number of real labels to construct a valid lower bound on operational risk at any time point. Notably, PPRM makes no assumptions about the underlying data distribution and provides finite-sample guarantees on false alarm probabilities. Empirical evaluations demonstrate that PPRM reliably detects harmful distribution shifts while effectively controlling false alarm rates across diverse tasks, including image classification, large language models, and telecommunications monitoring.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Programming
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We study the problem of monitoring model performance in dynamic environments where labeled data are limited. To this end, we propose prediction-powered risk monitoring (PPRM), a semi-supervised risk-monitoring approach based on prediction-powered inference (PPI). PPRM constructs anytime-valid lower bounds on the running risk by combining synthetic labels with a small set of true labels. Harmful shifts are detected via a threshold-based comparison with an upper bound on the nominal risk, satisfying assumption-free finite-sample guarantees in the probability of false alarm. We demonstrate the effectiveness of PPRM through extensive experiments on image classification, large language model (LLM), and telecommunications monitoring tasks.

Problem

• risk monitoring
• distribution shift
• limited labeled data
• model deployment
• performance monitoring

Innovation

• prediction-powered inference
• risk monitoring
• distribution shift detection
• semi-supervised learning
• anytime-valid bounds