ADELIA: Automatic Differentiation for Efficient Laplace Inference Approximations

2026-05-07

AI Summary

This work addresses the scalability limitations of traditional Integrated Nested Laplace Approximation (INLA) methods, which rely on finite-difference approximations for gradient computation—a strategy whose computational cost and energy consumption grow prohibitively with the dimensionality of hyperparameters, hindering application to large-scale spatiotemporal Bayesian inference. To overcome this bottleneck, the study introduces, for the first time, structure-aware reverse-mode automatic differentiation into the INLA framework, leveraging model sparsity to design a multi-GPU-efficient backpropagation algorithm tailored for high-dimensional sparse latent Gaussian models. Evaluated on a real-world air pollution monitoring task involving 1.9 million latent variables, the proposed approach achieves 4.2–7.9× speedup per gradient evaluation and exhibits markedly improved convergence stability. Moreover, it reduces energy consumption by 5–8× compared to an optimized finite-difference baseline.

Abstract

Spatio-temporal Bayesian inference drives environmental and health sciences using latent Gaussian models. Integrated Nested Laplace Approximations (INLA) enable inference for these models at HPC scale but rely on derivative-based optimization over $d$ hyperparameters. State-of-the-art INLA implementations approximate derivatives via central finite differences (FD), requiring $2d{+}1$ evaluations. These evaluations are embarrassingly parallel, but total work and energy grow with $d$, limiting time-to-solution under fixed budgets. Reverse-mode automatic differentiation (AD) computes exact gradients independently of $d$, but its efficient application to INLA's structured-sparse kernels is an open challenge. We present ADELIA, the first AD-enabled INLA implementation with a structure-exploiting multi-GPU backward pass leveraging model sparsity. We evaluate ADELIA on ten benchmark models, including real-world air-pollution monitoring. We achieve $4.2$--$7.9\times$ per-gradient speedups and reliable convergence on production-scale models with up to 1.9M latent variables, where FD struggles. Even when scaled to 16--32 GPUs to match ADELIA's wall-clock time, FD consumes $5$--$8\times$ more energy.

Problem

• Integrated Nested Laplace Approximations
• automatic differentiation
• Bayesian inference
• hyperparameter optimization
• computational efficiency

Innovation

• Automatic Differentiation
• INLA
• Structured Sparsity
• Multi-GPU
• Bayesian Inference