Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference

2026-01-30

AI Summary

This work addresses the challenge of efficiently leveraging GPU parallelism for Bayesian inference and model evidence computation under complex, multimodal posterior distributions. We propose the first GPU-accelerated, vectorized nested sampling algorithm, which integrates Hit-and-Run slice sampling for constrained updates and introduces an approximately optimal slice width rule to enhance high-dimensional performance and parallel efficiency. The method substantially improves the robustness and scalability of evidence estimation and posterior sampling in multimodal settings. Empirical evaluations on synthetic multimodal targets, high-dimensional Bayesian inference problems, and Gaussian process hyperparameter marginalization demonstrate superior accuracy and sample quality compared to state-of-the-art annealed sequential Monte Carlo (SMC) approaches.

Technology Category

• Machine Learning: Bayesian Learning
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Model comparison and calibrated uncertainty quantification often require integrating over parameters, but scalable inference can be challenging for complex, multimodal targets. Nested Sampling is a robust alternative to standard MCMC, yet its typically sequential structure and hard constraints make efficient accelerator implementations difficult. This paper introduces Nested Slice Sampling (NSS), a GPU-friendly, vectorized formulation of Nested Sampling that uses Hit-and-Run Slice Sampling for constrained updates. A tuning analysis yields a simple near-optimal rule for setting the slice width, improving high-dimensional behavior and making per-step compute more predictable for parallel execution. Experiments on challenging synthetic targets, high dimensional Bayesian inference, and Gaussian process hyperparameter marginalization show that NSS maintains accurate evidence estimates and high-quality posterior samples, and is particularly robust on difficult multimodal problems where current state-of-the-art methods such as tempered SMC baselines can struggle. An open-source implementation is released to facilitate adoption and reproducibility.

Problem

• Nested Sampling
• GPU acceleration
• multimodal inference
• Bayesian model comparison
• uncertainty quantification

Innovation

• Nested Slice Sampling
• GPU acceleration
• vectorized inference
• constrained sampling
• evidence estimation