Implicit Variational Rejection Sampling

2026-06-12

AI Summary

Traditional variational inference is often constrained by the mean-field assumption, limiting its ability to accurately approximate complex posterior distributions; meanwhile, existing implicit methods still incur bias due to architectural choices in neural networks. This work proposes the Implicit Resampling Evidence Lower Bound (IR-ELBO), which integrates a rejection sampling mechanism into implicit variational inference. Specifically, it employs a flexible neural implicit proposal distribution and leverages a discriminator to estimate the density ratio between this proposal and the true posterior, enabling unbiased resampling. The resulting framework yields a tighter variational lower bound, substantially enhancing both the expressiveness and accuracy of posterior approximation. Empirical evaluations demonstrate that the proposed method consistently outperforms conventional and state-of-the-art implicit variational approaches across multiple benchmarks.

Abstract

Variational Inference (VI) is a fundamental inference technique in Bayesian machine learning for approximating complex posterior distributions. Traditional VI often relies on the mean-field factorization, which can inadequately capture true posterior complexity. Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility. However, the practical constraints of neural network architectures still produces inaccuracies. In this paper, we propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation. Our method uses neural networks to construct implicit proposal distributions, and rejection sampling with a discriminator network that estimates the density ratio between the implicit proposal and the true posterior for refining the approximation. Towards this end, we introduce the Implicit Resampling Evidence Lower Bound (IR-ELBO) as a metric to characterize the resampled distribution's quality and derive a tighter variational lower bound. Experimental results demonstrate that our method outperforms traditional variational inference techniques.

Problem

• Variational Inference
• Posterior Approximation
• Implicit Distributions
• Rejection Sampling
• Bayesian Inference

Innovation

• Implicit Variational Inference
• Rejection Sampling
• Density Ratio Estimation
• Neural Posterior Approximation
• Tighter ELBO