Corrected Integrated Laplace Approximation for Bayesian Inference in Latent Gaussian Models

2026-05-19

AI Summary

This work addresses the significant bias introduced by Integrated Laplace Approximation (ILA) in latent-variable Gaussian models, which often leads to posterior distributions that deviate substantially from the true posterior and degrade downstream task performance. To mitigate this issue, the authors propose a correction framework based on importance sampling that integrates pseudo-marginalization with quasi-Monte Carlo methods within an automatic differentiation environment, enabling gradient-based hyperparameter inference. The approach innovatively combines randomized quasi-Monte Carlo with Hamiltonian Monte Carlo to construct an error-correction mechanism provably convergent to the true posterior. Experimental results demonstrate that the proposed method substantially reduces approximation error across a range of practical models and significantly enhances the accuracy of Bayesian inference.

Abstract

Latent Gaussian models (LGMs) are a popular class of Bayesian hierarchical models that include Gaussian processes, as well as certain spatial models and mixed-effect models. Efficient Bayesian inference of LGMs often requires marginalizing out the latent variables. For LGMs with a non-Gaussian likelihood, exact marginalization is not possible and a popular approach is to do approximate marginalization with an integrated Laplace approximation (ILA). Using ILA produces an approximate posterior which, in some settings, can differ significantly from the correct posterior, which impacts downstream applications. We propose an importance sampling scheme to correct the error introduced by ILA. By increasing the number of samples in importance sampling, the posterior with ILA converges to the correct posterior. This idea is realized with various techniques, including pseudo-marginalization, quasi-Monte Carlo and randomized quasi-Monte Carlo. We implement our methods in an automatic differentiation framework to support gradient-based algorithms when doing inference on the hyperparameters. For the latter, we specifically consider the use of Hamiltonian Monte Carlo. We demonstrate the benefits of reduced error in various applied models.

Problem

• Latent Gaussian models
• Bayesian inference
• Integrated Laplace Approximation
• posterior approximation error
• non-Gaussian likelihood

Innovation

• importance sampling
• integrated Laplace approximation
• latent Gaussian models
• quasi-Monte Carlo
• Hamiltonian Monte Carlo