The Gaussian-Head OFL Family: One-Shot Federated Learning from Client Global Statistics

2026-02-01

AI Summary

Traditional federated learning suffers from high communication overhead and privacy risks, while existing single-round methods often rely on public data, homogeneous models, or additional uploaded information, limiting their practicality. This work proposes Gaussian Head One-Round Federated Learning (GH-OFL), a framework that operates under strict no-data-sharing conditions: clients upload only sufficient statistics—counts, first-order, and second-order moments—of class-conditional embeddings, enabling the server to construct a classification head in a single aggregation round. GH-OFL uniquely integrates class-conditional Gaussian assumptions (as in Naive Bayes, LDA, or QDA), a FisherMix linear head synthesized via Fisher subspace exemplars, and a lightweight Proto-Hyper residual head based on knowledge distillation. The method supports heterogeneous client models, requires no public data, and achieves state-of-the-art accuracy and robustness under strong non-IID settings.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Classical Federated Learning relies on a multi-round iterative process of model exchange and aggregation between server and clients, with high communication costs and privacy risks from repeated model transmissions. In contrast, one-shot federated learning (OFL) alleviates these limitations by reducing communication to a single round, thereby lowering overhead and enhancing practical deployability. Nevertheless, most existing one-shot approaches remain either impractical or constrained, for example, they often depend on the availability of a public dataset, assume homogeneous client models, or require uploading additional data or model information. To overcome these issues, we introduce the Gaussian-Head OFL (GH-OFL) family, a suite of one-shot federated methods that assume class-conditional Gaussianity of pretrained embeddings. Clients transmit only sufficient statistics (per-class counts and first/second-order moments) and the server builds heads via three components: (i) Closed-form Gaussian heads (NB/LDA/QDA) computed directly from the received statistics; (ii) FisherMix, a linear head with cosine margin trained on synthetic samples drawn in an estimated Fisher subspace; and (iii) Proto-Hyper, a lightweight low-rank residual head that refines Gaussian logits via knowledge distillation on those synthetic samples. In our experiments, GH-OFL methods deliver state-of-the-art robustness and accuracy under strong non-IID skew while remaining strictly data-free.

Problem

• Federated Learning
• One-Shot Learning
• Communication Efficiency
• Privacy
• Non-IID Data

Innovation

• One-shot Federated Learning
• Gaussian-Head OFL
• Data-Free
• Non-IID Robustness
• Sufficient Statistics