On Continuous Monitoring of Risk Violations under Unknown Shift

2025-06-19

AI Summary

Real-world machine learning systems operate under dynamic data distribution shifts, rendering conventional risk control methods—predicated on static distributional assumptions—ineffective and incapable of online monitoring for decision risk violations. To address this, we propose the first sequential testing framework grounded in the “betting” paradigm, which strictly controls the false alarm rate (≤ α) under arbitrary, unknown distributional shifts—without assuming prior knowledge of drift type or underlying distributions. Our approach unifies betting-based hypothesis testing, risk-bound modeling, and online streaming statistical inference to enable real-time, robust monitoring of model risk. Extensive experiments demonstrate that the method achieves high sensitivity in detecting risk violations across diverse drift scenarios, while simultaneously delivering rigorous statistical guarantees in both anomaly detection and conformal prediction tasks.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Sequential Decision Making
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Machine learning systems deployed in the real world must operate under dynamic and often unpredictable distribution shifts. This challenges the validity of statistical safety assurances on the system's risk established beforehand. Common risk control frameworks rely on fixed assumptions and lack mechanisms to continuously monitor deployment reliability. In this work, we propose a general framework for the real-time monitoring of risk violations in evolving data streams. Leveraging the'testing by betting'paradigm, we propose a sequential hypothesis testing procedure to detect violations of bounded risks associated with the model's decision-making mechanism, while ensuring control on the false alarm rate. Our method operates under minimal assumptions on the nature of encountered shifts, rendering it broadly applicable. We illustrate the effectiveness of our approach by monitoring risks in outlier detection and set prediction under a variety of shifts.

Problem

• Monitor risk violations in dynamic data streams
• Detect bounded risk violations with false alarm control
• Ensure reliability under unknown distribution shifts

Innovation

• Real-time risk monitoring in evolving data streams
• Sequential hypothesis testing for risk violations
• Minimal assumptions on encountered distribution shifts