PAC Off-Policy Prediction of Contextual Bandits

2025-07-22

AI Summary

This paper addresses off-policy evaluation in contextual bandits—estimating a target policy’s performance with reliable confidence intervals from historical data, under unknown and mismatched behavior policies. We propose a novel algorithm integrating conformal prediction with off-policy estimation, the first to achieve PAC (Probably Approximately Correct) guarantees under conditional coverage: it rigorously ensures the nominal coverage probability is met or exceeded in finite samples. Our theoretical analysis provides both finite-sample bounds and asymptotic properties, substantially strengthening statistical guarantees over existing methods. Experiments demonstrate that our approach achieves high empirical coverage while markedly improving interval width; the tight alignment between theoretical guarantees and empirical performance underscores its practical reliability. The method is particularly suited for high-stakes decision-making domains such as healthcare and finance, where rigorous uncertainty quantification is critical.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper investigates off-policy evaluation in contextual bandits, aiming to quantify the performance of a target policy using data collected under a different and potentially unknown behavior policy. Recently, methods based on conformal prediction have been developed to construct reliable prediction intervals that guarantee marginal coverage in finite samples, making them particularly suited for safety-critical applications. To further achieve coverage conditional on a given offline data set, we propose a novel algorithm that constructs probably approximately correct prediction intervals. Our method builds upon a PAC-valid conformal prediction framework, and we strengthen its theoretical guarantees by establishing PAC-type bounds on coverage. We analyze both finite-sample and asymptotic properties of the proposed method, and compare its empirical performance with existing methods in simulations.

Problem

• Evaluate target policy performance using off-policy data
• Construct reliable prediction intervals with marginal coverage
• Achieve conditional coverage on given offline datasets

Innovation

• PAC-valid conformal prediction framework
• Conditional coverage on offline data
• Finite-sample and asymptotic theoretical guarantees