Onsager-Machlup Posterior Transport for Deep Gaussian Processes

2026-05-22

AI Summary

This work addresses the computational bottleneck in sparse inference for deep Gaussian processes (DGPs) by proposing OM-Path, a novel approach that frames posterior inference over inducing variables as a posterior transport problem. Specifically, OM-Path employs a deterministic sampler to map a tractable reference measure onto the inducing variable posterior and regularizes this mapping using a path prior derived from Doob bridge diffusion. By integrating Song’s probability flow ODE with the forward SDE of Doubly Reparameterized Variational Inference (DBVI), the method leverages the Onsager–Machlup functional as a path regularizer to obtain a closed-form reference drift without requiring score matching, while connecting small-noise MAP paths under finite noise. Experiments demonstrate that OM-Path significantly outperforms DBVI on the two largest UCI regression benchmarks—power and protein—achieving state-of-the-art results in both negative log-likelihood (NLL) and RMSE, and performs comparably or slightly worse on smaller datasets.

Abstract

Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs). Existing methods either fit an explicit density $q_φ(\bU)$ by an ELBO (DSVI, IPVI, DDVI, DBVI) or sample by MCMC (SGHMC). We instead frame DGP inference as \emph{posterior transport}: learn a deterministic sampler that maps a tractable reference measure to posterior-relevant inducing variables, regularised by a path prior derived from the Doob-bridged reference diffusion. Our realisation, \textbf{OM-Path} (formally FBVI-bridge-Path), uses Song's probability-flow ODE applied to DBVI's Doob-bridged forward SDE; the reference drift is closed-form from the bridge marginal coefficients (no score matching) and the path regulariser is the \textbf{Onsager--Machlup action}. At the finite-$ε$ value used at training, the objective is the negative log unnormalised density of a tempered Doob-bridge path posterior, and Theorem 1 identifies it with the same posterior's small-noise MAP path via the Freidlin--Wentzell LDP. Two strict path-space ELBO variants on the same bridge backbone (FFJORD log-det; OM-regularised CNF) are derived as ablations. Under a matched-seed paired Wilcoxon test against DBVI on seven UCI regression benchmarks, OM-Path delivers statistically significant wins on the two largest datasets (\textit{power}: $p\!=\!0.014$, NLL $\mathbf{0.012}$ matching the DSVI baseline of $0.017$; \textit{protein}: $p\!=\!0.002$, RMSE $\mathbf{0.716}$ vs.\ $0.764$, NLL $\mathbf{1.086}$ vs.\ $1.149$), statistical ties on \textit{yacht} / \textit{qsar}, and concedes \textit{boston} / \textit{energy} / \textit{concrete} to DBVI on small-$N$ noisy data. The strict-ELBO variants do not clear DBVI on any UCI metric: in this regime, reducing the variance of the path objective dominates exact-density tracking.

Problem

• Deep Gaussian Processes
• approximate inference
• inducing variables
• posterior transport
• computational bottleneck

Innovation

• Posterior Transport
• Onsager-Machlup Action
• Deep Gaussian Processes
• Doob Bridge
• Probability Flow ODE