Drift-to-Action Controllers: Budgeted Interventions with Online Risk Certificates

2026-03-09

AI Summary

This work addresses the lack of safe and controllable response mechanisms in machine learning systems under distribution shift by proposing a monitoring framework grounded in constrained safe decision-making. The approach identifies shift types at the perception layer and, for the first time, introduces an online risk certificate mechanism that provides a valid, anytime upper bound on current risk, thereby triggering tiered intervention strategies. Key technical components include unlabeled signal modeling, delayed label querying, test-time adaptation, and active abstention. Experiments on WILDS Camelyon17, DomainNet, and synthetic data streams demonstrate that the method achieves near-zero safety violations and rapid recovery, significantly outperforming existing baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Large-scale security measurements

Abstract

Deployed machine learning systems face distribution drift, yet most monitoring pipelines stop at alarms and leave the response underspecified under labeling, compute, and latency constraints. We introduce Drift2Act, a drift-to-action controller that treats monitoring as constrained decision-making with explicit safety. Drift2Act combines a sensing layer that maps unlabeled monitoring signals to a belief over drift types with an active risk certificate that queries a small set of delayed labels from a recent window to produce an anytime-valid upper bound $U_t(\delta)$ on current risk. The certificate gates operation: if $U_t(\delta) \le \tau$, the controller selects low-cost actions (e.g., recalibration or test-time adaptation); if $U_t(\delta)>\tau$, it activates abstain/handoff and escalates to rollback or retraining under cooldowns. In a realistic streaming protocol with label delay and explicit intervention costs, Drift2Act achieves near-zero safety violations and fast recovery at moderate cost on WILDS Camelyon17, DomainNet, and a controlled synthetic drift stream, outperforming alarm-only monitoring, adapt-always adaptation, schedule-based retraining, selective prediction alone, and an ablation without certification. Overall, online risk certification enables reliable drift response and reframes monitoring as decision-making with safety.

Problem

• distribution drift
• budgeted interventions
• online risk certification
• safety constraints
• machine learning monitoring

Innovation

• drift-to-action control
• online risk certification
• constrained decision-making
• distribution drift
• active monitoring