Importance Weighted Variational Inference without the Reparameterization Trick

2026-02-01

AI Summary

This work addresses a critical limitation in importance-weighted variational inference: existing REINFORCE-type gradient estimators, such as VIMCO, lack theoretical guarantees and suffer from a signal-to-noise ratio (SNR) that vanishes as the number of samples increases, leading to optimization failure. The paper presents the first systematic analysis of REINFORCE gradients in this setting and introduces a novel estimator, VIMCO-$\star$, which—without relying on reparameterization—ensures that the SNR grows with the square root of the number of samples, thereby avoiding SNR collapse. Empirical results demonstrate that VIMCO-$\star$ significantly outperforms current methods in complex scenarios where reparameterization gradients are inapplicable, offering both theoretical rigor and practical efficacy.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Importance weighted variational inference (VI) approximates densities known up to a normalizing constant by optimizing bounds that tighten with the number of Monte Carlo samples $N$. Standard optimization relies on reparameterized gradient estimators, which are well-studied theoretically yet restrict both the choice of the data-generating process and the variational approximation. While REINFORCE gradient estimators do not suffer from such restrictions, they lack rigorous theoretical justification. In this paper, we provide the first comprehensive analysis of REINFORCE gradient estimators in importance weighted VI, leveraging this theoretical foundation to diagnose and resolve fundamental deficiencies in current state-of-the-art estimators. Specifically, we introduce and examine a generalized family of variational inference for Monte Carlo objectives (VIMCO) gradient estimators. We prove that state-of-the-art VIMCO gradient estimators exhibit a vanishing signal-to-noise ratio (SNR) as $N$ increases, which prevents effective optimization. To overcome this issue, we propose the novel VIMCO-$\star$ gradient estimator and show that it averts the SNR collapse of existing VIMCO gradient estimators by achieving a $\sqrt{N}$ SNR scaling instead. We demonstrate its superior empirical performance compared to current VIMCO implementations in challenging settings where reparameterized gradients are typically unavailable.

Problem

• importance weighted variational inference
• REINFORCE gradient estimator
• signal-to-noise ratio
• VIMCO
• gradient estimation

Innovation

• Importance Weighted Variational Inference
• REINFORCE Gradient Estimator
• Signal-to-Noise Ratio
• VIMCO
• Reparameterization Trick