Exploring Pareto smoothing in sequential Monte Carlo

2026-06-24

AI Summary

This study addresses the high computational cost of Markov chain Monte Carlo (MCMC) steps commonly employed in sequential Monte Carlo (SMC) and approximate Bayesian computation (ABC)-SMC to control the variance of importance weights. For the first time, it systematically integrates Pareto smoothed importance sampling (PSIS) into the SMC framework, leveraging a fitted generalized Pareto distribution to adjust the tails of the importance weights and thereby reduce their variance, with the aim of diminishing reliance on MCMC. However, empirical analysis reveals that because SMC inherently mitigates weight degeneracy through its sequence of intermediate target distributions, the additional variance reduction offered by PSIS is limited. This finding challenges the presumed necessity of PSIS in this context and provides new theoretical and empirical insights into strategies for stabilizing importance weights in SMC algorithms.

Abstract

A popular technique for reducing the variance of importance sampling (IS) estimators is to modify the weights of some importance points. One approach is to truncate the largest weights, which reduces variance but can introduce substantial bias. Pareto smoothed importance sampling (PSIS), by contrast, reduces the variance of the weights by fitting a generalised Pareto distribution to the upper tail of the weight distribution and replacing the weights in this tail with the corresponding expected quantiles from the fitted distribution. PSIS can therefore also reduce variance, but typically with less bias, and has been used successfully in IS-based approximations for Bayesian cross-validation. This paper explores the use of PSIS steps within sequential Monte Carlo (SMC) samplers, with a particular focus on approximate Bayesian computation (ABC)-SMC algorithms, where we aim to use Pareto smoothing to reduce the use of Markov chain Monte Carlo (MCMC) moves, each of which requires simulation from a model that is often computationally expensive. Our empirical investigation suggests that there are only minimal benefits to using Pareto smoothing in SMC, since the variance reduction through using a sequence of targets dominates the impact of the weight adjustment.

Problem

• sequential Monte Carlo
• Pareto smoothing
• approximate Bayesian computation
• importance sampling
• MCMC moves

Innovation

• Pareto smoothing
• sequential Monte Carlo
• importance sampling
• approximate Bayesian computation
• variance reduction