Online Conformal Prediction with Corrupted Feedback

2026-05-19

AI Summary

This work addresses the degradation of coverage guarantees in online conformal prediction when feedback is corrupted by noise, communication failures, or adversarial attacks. It presents the first systematic modeling of arbitrary binary feedback corruption and provides explicit miscoverage bounds under two distinct error models: independent random flips and memory-bounded adversarial errors. To mitigate the impact of corrupted feedback, the paper introduces two robust mechanisms—a threshold-based feedback filtering scheme and an active compensation strategy—effectively preserving predictive reliability. The proposed approach integrates online conformal prediction with robust statistical learning, making it suitable for non-stationary sequential environments. Empirical evaluations on real-world datasets demonstrate substantial improvements over existing baselines, achieving better calibration and significantly smaller prediction sets.

Abstract

Modern artificial intelligence systems require calibrated uncertainty estimates that remain reliable in sequential and non-stationary environments. Online conformal prediction (OCP) addresses this challenge through adaptively updated prediction sets that provide deterministic long-run miscoverage guarantees. These guarantees, however, hinge on the assumption of perfect feedback about the coverage of past prediction sets. In practice, the observed miscoverage indicator may be corrupted by noise, communication failures, or adversarial manipulation, which can severely degrade OCP's calibration guarantees. In this paper, we study OCP under corrupted feedback. We first model feedback corruption as an arbitrary binary flip sequence, and analyze how feedback corruption affects and degrades the miscoverage performance of standard OCP. We then propose two robust schemes: robust OCP via filtering, which leverages the structural properties of the predicted threshold to filter corrupted feedback, and robust OCP via active compensation, which incorporates an active compensation mechanism to mitigate the effect of corrupted feedback. For both methods, we establish explicit miscoverage guarantees, which are further specialized for an independent stochastic flip model and for an arbitrary error model with memory bounds. Experiments on real-world datasets validate the proposed approach, showing markedly improved calibration and significantly smaller prediction sets compared with baseline OCP methods under corrupted feedback.

Problem

• Online Conformal Prediction
• Corrupted Feedback
• Uncertainty Calibration
• Miscoverage Guarantee
• Non-stationary Environments

Innovation

• Online Conformal Prediction
• Corrupted Feedback
• Robust Calibration
• Active Compensation
• Miscoverage Guarantee