Sampling as Bandits: Evaluation-Efficient Design for Black-Box Densities

2025-09-01

AI Summary

For expensive black-box target density sampling, this paper proposes an active sampling method framed within the multi-armed bandit (MAB) paradigm. Unlike conventional approaches that optimize a proposal distribution, our method models sample location selection as a sequential decision-making process, jointly leveraging a Gaussian process surrogate model and space-filling criteria to adaptively select evaluation points with maximal information gain. To our knowledge, this is the first work to directly apply MAB for sampling point scheduling—bypassing explicit distribution modeling and substantially reducing the number of target function evaluations. Experiments demonstrate superior performance over state-of-the-art importance sampling methods on multimodal and heavy-tailed distributions. In Bayesian inference tasks, our approach achieves higher approximation accuracy with significantly fewer evaluations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Active Learning
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We introduce bandit importance sampling (BIS), a new class of importance sampling methods designed for settings where the target density is expensive to evaluate. In contrast to adaptive importance sampling, which optimises a proposal distribution, BIS directly designs the samples through a sequential strategy that combines space-filling designs with multi-armed bandits. Our method leverages Gaussian process surrogates to guide sample selection, enabling efficient exploration of the parameter space with minimal target evaluations. We establish theoretical guarantees on convergence and demonstrate the effectiveness of the method across a broad range of sampling tasks. BIS delivers accurate approximations with fewer target evaluations, outperforming competing approaches across multimodal, heavy-tailed distributions, and real-world applications to Bayesian inference of computationally expensive models.

Problem

• Efficient sampling for expensive black-box density evaluation
• Reducing target evaluations in multimodal and heavy-tailed distributions
• Improving Bayesian inference for computationally expensive models

Innovation

• Bandit importance sampling for expensive evaluations
• Combines space-filling designs with multi-armed bandits
• Uses Gaussian process surrogates to guide sampling