FedBEns: One-Shot Federated Learning based on Bayesian Ensemble

2025-03-19

AI Summary

To address the limited generalization capability of global models in one-shot federated learning (FL) under single-round communication, this paper proposes a multimodal Bayesian ensemble method. It models the multimodality of client-wise local losses from a Bayesian perspective, constructs a mixture-of-posteriors distribution for each client via Laplace approximation, and infers the global model through weighted aggregation of these local posterior mixtures at the server. This work is the first to introduce multimodal posterior modeling into one-shot FL, overcoming the limitations of conventional unimodal approximations. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method significantly outperforms state-of-the-art one-shot FL algorithms, achieving average test accuracy improvements of 2.3–5.1 percentage points. The results validate that multimodal posterior modeling plays a critical role in enhancing the robustness and generalization performance of the global model.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

One-Shot Federated Learning (FL) is a recent paradigm that enables multiple clients to cooperatively learn a global model in a single round of communication with a central server. In this paper, we analyze the One-Shot FL problem through the lens of Bayesian inference and propose FedBEns, an algorithm that leverages the inherent multimodality of local loss functions to find better global models. Our algorithm leverages a mixture of Laplace approximations for the clients' local posteriors, which the server then aggregates to infer the global model. We conduct extensive experiments on various datasets, demonstrating that the proposed method outperforms competing baselines that typically rely on unimodal approximations of the local losses.

Problem

• One-Shot Federated Learning for global model
• Bayesian inference to improve model accuracy
• Mixture of Laplace approximations for aggregation

Innovation

• Bayesian inference for One-Shot FL
• Mixture of Laplace approximations
• Aggregates local posteriors globally