Recursive variational Gaussian approximation with the Whittle likelihood for linear non-Gaussian state space models

2024-06-23

AI Summary

Parameter inference in linear non-Gaussian state-space models faces computational bottlenecks due to intractable integrals over latent states in the likelihood. To address this, we propose a recursive variational Gaussian approximation method grounded in the Whittle likelihood. This work is the first to embed the Whittle likelihood into a recursive variational Bayesian framework, enabling sequential closed-form updates of variational parameters directly in the frequency domain. We derive analytical expressions for both the gradient and Hessian of the variational objective, balancing estimation accuracy with scalability. Empirical evaluation across multiple model classes shows that our method achieves posterior approximation quality comparable to exact methods such as Hamiltonian Monte Carlo (HMC), while delivering 10–100× speedup in computation time. Moreover, it supports real-time inference on long time-series data—unattainable for many existing approaches—thereby advancing scalable Bayesian inference for non-Gaussian dynamical systems.

Technology Category

• Machine Learning: Bayesian Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Parameter inference for linear and non-Gaussian state space models is challenging because the likelihood function contains an intractable integral over the latent state variables. While Markov chain Monte Carlo (MCMC) methods provide exact samples from the posterior distribution as the number of samples go to infinity, they tend to have high computational cost, particularly for observations of a long time series. Variational Bayes (VB) methods are a useful alternative when inference with MCMC methods is computationally expensive. VB methods approximate the posterior density of the parameters by a simple and tractable distribution found through optimisation. In this paper, we propose a novel sequential variational Bayes approach that makes use of the Whittle likelihood for computationally efficient parameter inference in this class of state space models. Our algorithm, which we call Recursive Variational Gaussian Approximation with the Whittle Likelihood (R-VGA-Whittle), updates the variational parameters by processing data in the frequency domain. At each iteration, R-VGA-Whittle requires the gradient and Hessian of the Whittle log-likelihood, which are available in closed form for a wide class of models. Through several examples using a linear Gaussian state space model and a univariate/bivariate non-Gaussian stochastic volatility model, we show that R-VGA-Whittle provides good approximations to posterior distributions of the parameters and is very computationally efficient when compared to asymptotically exact methods such as Hamiltonian Monte Carlo.

Problem

• Efficient parameter inference in non-Gaussian state space models
• Overcoming intractable integrals in likelihood functions
• Reducing computational cost compared to MCMC methods

Innovation

• Sequential VB approach with Whittle likelihood
• Recursive Variational Gaussian Approximation algorithm
• Frequency domain data processing for efficiency