Score-Based Martingale Posteriors for Deep Neural Networks

2026-06-14

AI Summary

Quantifying uncertainty in deep neural networks is often hindered by the prohibitive computational cost of traditional Bayesian approaches such as Markov chain Monte Carlo (MCMC). This work proposes a novel paradigm grounded in the Score-driven Martingale Posterior (SMP) framework, which circumvents the need for Markov chains by recursively constructing a martingale sequence over the parameter space via score-function-guided stochastic gradient ascent. The limiting distribution of this sequence is then efficiently sampled to approximate the posterior. By extending martingale posterior theory to large-scale deep learning settings, the method achieves uncertainty estimates comparable in quality to state-of-the-art Monte Carlo techniques while offering substantially improved computational efficiency.

Abstract

In this paper we investigate the efficacy of the score-based martingale posteriors (SMP) (Cui & Walker, 2025; Fong et al., 2023) in the context of modern and large-scale machine learning problems and its potential for meaningful uncertainty quantification. SMPs work with a stochastic gradient ascent-type recursion on the parameter space of stochastic models and construct a martingale on the parameter space. Under simple mathematical assumptions, the recursion can be built so that the parameters form a martingale sequence which possesses a limiting, in time, random variable, the latter of which can be simulated very quickly, in contrast to Monte Carlo-based methods such as Markov chain Monte Carlo. In this expository paper we explore the SMP for inferring the parameters of deep neural networks (DNNs) and, where feasible, compare our results to the state-of-the-art Monte Carlo methods aimed at inferring conventional Bayesian posteriors.

Problem

• uncertainty quantification
• deep neural networks
• Bayesian inference
• large-scale machine learning
• posterior inference

Innovation

• score-based martingale posteriors
• deep neural networks
• uncertainty quantification
• stochastic gradient ascent
• Bayesian inference