Stochastic Online Conformal Prediction with Semi-Bandit Feedback

2024-05-22

arXiv.org

AI Summary

This work addresses the challenge of dynamically constructing statistically valid prediction sets under semi-bandit feedback in online learning—where the true label is observable only if it falls within the predicted set. We introduce conformal prediction to the stochastic online setting for the first time. Our method proposes the first adaptive algorithm with a sublinear regret bound, eliminating reliance on an i.i.d. calibration set. By integrating stochastic optimization and online learning theory, we design a confidence-set-based dynamic threshold update mechanism that simultaneously ensures coverage guarantees and minimizes regret. Empirical evaluation across document retrieval, image classification, and auction pricing demonstrates significant improvements over baselines, validating both statistical validity and practical utility.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Conformal prediction has emerged as an effective strategy for uncertainty quantification by modifying a model to output sets of labels instead of a single label. These prediction sets come with the guarantee that they contain the true label with high probability. However, conformal prediction typically requires a large calibration dataset of i.i.d. examples. We consider the online learning setting, where examples arrive over time, and the goal is to construct prediction sets dynamically. Departing from existing work, we assume semi-bandit feedback, where we only observe the true label if it is contained in the prediction set. For instance, consider calibrating a document retrieval model to a new domain; in this setting, a user would only be able to provide the true label if the target document is in the prediction set of retrieved documents. We propose a novel conformal prediction algorithm targeted at this setting, and prove that it obtains sublinear regret compared to the optimal conformal predictor. We evaluate our algorithm on a retrieval task, an image classification task, and an auction price-setting task, and demonstrate that it empirically achieves good performance compared to several baselines.

Problem

• Online conformal prediction with semi-bandit feedback
• Dynamic prediction set construction without i.i.d. data
• Sublinear regret compared to optimal conformal predictor

Innovation

• Online conformal prediction with semi-bandit feedback
• Dynamic prediction sets for uncertainty quantification
• Sublinear regret compared to optimal conformal predictor