🤖 AI Summary
To address the high computational cost of sequential Monte Carlo (SMC) arising from particle degeneracy, mode collapse, and the need for large particle counts, this paper introduces the Persistent Sampling (PS) framework. PS retains and reweights historical particles across iterations to construct a progressively refined mixture importance distribution, enabling particle reuse without additional likelihood evaluations—a first in SMC literature. The method integrates multiple importance sampling, mixture-based resampling over historical distributions, cached likelihood weight updates, and adaptive MCMC transition kernels. Crucially, it reduces variance in marginal likelihood estimation without increasing likelihood evaluations and facilitates kernel optimization using large, low-autocorrelation particle pools. Experiments on high-dimensional Gaussian mixtures, hierarchical models, and non-convex targets demonstrate that PS achieves lower mean-squared error in posterior expectations and evidence estimation—and lower computational cost—than standard SMC and state-of-the-art recycling and waste-free variants.
📝 Abstract
Sequential Monte Carlo (SMC) samplers are powerful tools for Bayesian inference but suffer from high computational costs due to their reliance on large particle ensembles for accurate estimates. We introduce persistent sampling (PS), an extension of SMC that systematically retains and reuses particles from all prior iterations to construct a growing, weighted ensemble. By leveraging multiple importance sampling and resampling from a mixture of historical distributions, PS mitigates the need for excessively large particle counts, directly addressing key limitations of SMC such as particle impoverishment and mode collapse. Crucially, PS achieves this without additional likelihood evaluations-weights for persistent particles are computed using cached likelihood values. This framework not only yields more accurate posterior approximations but also produces marginal likelihood estimates with significantly lower variance, enhancing reliability in model comparison. Furthermore, the persistent ensemble enables efficient adaptation of transition kernels by leveraging a larger, decorrelated particle pool. Experiments on high-dimensional Gaussian mixtures, hierarchical models, and non-convex targets demonstrate that PS consistently outperforms standard SMC and related variants, including recycled and waste-free SMC, achieving substantial reductions in mean squared error for posterior expectations and evidence estimates, all at reduced computational cost. PS thus establishes itself as a robust, scalable, and efficient alternative for complex Bayesian inference tasks.