Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification

2026-05-12

AI Summary

This work addresses the gap between traditional Bayesian inference—which focuses on parameter posteriors—and practical needs that center on predictive distributions and their uncertainties. The authors propose Self-Supervised Laplace Approximation (SSLA), a framework that bypasses explicit parameter posterior computation by directly approximating the posterior predictive distribution through refitting the model on self-generated predictive data. This approach enables deterministic, sampling-free uncertainty quantification, flexibly accommodates diverse priors, and supports sensitivity analysis. An efficient variant, Approximate SSLA (ASSLA), is also introduced. Empirical evaluations across a range of models—from linear regressors to Bayesian neural networks—demonstrate that SSLA consistently outperforms classical Laplace approximation in both synthetic and real-world regression tasks, achieving superior predictive calibration while maintaining computational efficiency.

Abstract

Approximate Bayesian inference typically revolves around computing the posterior parameter distribution. In practice, however, the main object of interest is often a model's predictions rather than its parameters. In this work, we propose to bypass the parameter posterior and focus directly on approximating the posterior predictive distribution. We achieve this by drawing inspiration from self-training within self-supervised and semi-supervised learning. Essentially, we quantify a Bayesian model's predictive uncertainty by refitting on self-predicted data. The idea is strikingly simple: If a model assigns high likelihood to self-predicted data, these predictions are of low uncertainty, and vice versa. This yields a deterministic, sampling-free approximation of the posterior predictive. The modular structure of our Self-Supervised Laplace Approximation (SSLA) further allows us to plug in different prior specifications, enabling classical Bayesian sensitivity (w.r.t. prior choice) analysis. In order to bypass expensive refitting, we further introduce an approximate version of SSLA, called ASSLA. We study (A)SSLA both theoretically and empirically in regression models ranging from Bayesian linear models to Bayesian neural networks. Across a wide array of regression tasks with simulated and real-world datasets, our methods outperform classical Laplace approximations in predictive calibration while remaining computationally efficient.

Problem

• Bayesian uncertainty quantification
• posterior predictive distribution
• self-supervised learning
• Laplace approximation
• predictive calibration

Innovation

• Self-Supervised Laplace Approximation
• Bayesian Uncertainty Quantification
• Posterior Predictive Approximation
• Self-Training
• Predictive Calibration