Federated Martingale Posterior Samping

2026-05-18

AI Summary

This work addresses the degradation in accuracy and calibration commonly observed in federated Bayesian neural networks due to the difficulty of specifying appropriate priors and likelihoods. To overcome this challenge without sharing local data, the authors propose a novel federated predictive Bayesian approach that integrates martingale posteriors, trainable data embeddings, and a federated learning framework. They introduce the first single-round, parallelized federated Markov posterior sampling protocol, wherein clients upload learned embeddings and the server performs centralized predictive sampling to recover parameter uncertainty—thereby preserving privacy and eliminating reliance on explicit prior distributions. Experimental results demonstrate that the method achieves performance nearly on par with centralized training on MNIST, CIFAR-10, and CIFAR-100, while significantly outperforming existing consensus-based baselines in terms of calibration.

Abstract

Federated Bayesian neural networks require fixing a prior on the model parameters together with a likelihood. Eliciting meaningful priors on the weight space of modern overparameterized models is notoriously difficult, and misspecification of either component can severely degrade accuracy and calibration. Motivated by the rapid progress of predictive models such as large language models, the martingale posterior, also known as predictive Bayes, replaces the prior--likelihood pair with a predictive distribution and recovers parameter uncertainty by repeatedly drawing predictive samples and refitting the model. A direct federated implementation, however, would require clients to share the local data sets. This letter proposes {federated martingale posterior} (FMP) sampling, a one-shot embarrassingly parallel protocol in which each client uploads a small set of trainable data embeddings and the server runs the predictive sampler centrally. Experiments on MNIST, CIFAR-10, and CIFAR-100 show that FMP closely matches the centralized counterpart and significantly improves calibration over consensus-style baselines.

Problem

• Federated Learning
• Bayesian Neural Networks
• Prior Specification
• Predictive Bayes
• Data Privacy

Innovation

• Federated Learning
• Martingale Posterior
• Predictive Bayes
• Bayesian Neural Networks
• Data Embeddings