Kernel weighted importance sampling for off-policy evaluation in contextual bandits

2026-07-16

AI Summary

This work addresses the challenges of accuracy and stability in off-policy evaluation under complex conditions such as behavior policy misspecification in offline contextual bandits. We propose a novel Kernel-Weighted Importance Sampling (Kernel-WIS) estimator that innovatively integrates kernel methods with importance sampling, combining the bounded stability of weighted importance sampling with the low-bias linear properties of ordinary importance sampling. Theoretical analysis establishes the asymptotic consistency of Kernel-WIS, and empirical results demonstrate its significant superiority over strong baselines—including standard weighted importance sampling—across a range of challenging scenarios.

Abstract

This article presents a novel estimator for performing off-policy evaluation using only offline data for contextual bandits. The proposed estimator, Kernel-WIS is demonstrated to be asymptotically consistent and to empirically outperform strong baselines (including vanilla weighted importance sampling), particularly under complex conditions including behaviour policy miss-specification. The benefit of Kernel-WIS is derived from combining the bounded property of vanilla weighted importance sampling with the linearity of vanilla importance sampling.

Problem

• off-policy evaluation
• contextual bandits
• importance sampling
• offline data
• behavior policy misspecification

Innovation

• Kernel-WIS
• off-policy evaluation
• contextual bandits
• importance sampling
• asymptotic consistency