Optimality in importance sampling: a gentle survey

2025-02-11

AI Summary

This paper addresses the optimal design of proposal densities in Monte Carlo importance sampling, particularly under challenging settings involving dynamic updates and noise—such as in Approximate Bayesian Computation (ABC) and policy evaluation in reinforcement learning. It provides the first unified theoretical analysis of the applicability boundaries of multiple optimality criteria—including minimum variance and KL-divergence minimization—while establishing a cross-framework evaluation framework that jointly ensures theoretical guarantees and empirically comparable performance. Methodologically, the work integrates variational inference, sequential importance resampling, and annealed posterior modeling to propose a multi-proposal adaptive mechanism. Key contributions are: (1) necessary and sufficient conditions for proposal optimality across frameworks, with convergence guarantees; (2) systematic empirical validation of trade-offs among model selection accuracy, noise robustness, and computational efficiency in adaptive proposal design; and (3) an open-source empirical benchmark enabling reproducible, standardized comparison of future proposal mechanisms.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The performance of the Monte Carlo sampling methods relies on the crucial choice of a proposal density. The notion of optimality is fundamental to design suitable adaptive procedures of the proposal density within Monte Carlo schemes. This work is an exhaustive review around the concept of optimality in importance sampling. Several frameworks are described and analyzed, such as the marginal likelihood approximation for model selection, the use of multiple proposal densities, a sequence of tempered posteriors, and noisy scenarios including the applications to approximate Bayesian computation (ABC) and reinforcement learning, to name a few. Some theoretical and empirical comparisons are also provided.

Problem

• Importance sampling optimality in Monte Carlo
• Proposal density adaptation for efficiency
• Applications in Bayesian computation and reinforcement learning

Innovation

• Optimal proposal density design
• Multiple proposal densities usage
• Tempered posteriors sequence application