One-shot Federated Learning Methods: A Practical Guide

2025-02-13

AI Summary

This work systematically analyzes performance bottlenecks in one-shot federated learning (OFL) arising from data and model heterogeneity. It proposes the first unified taxonomy for OFL methods, encompassing model aggregation optimization, client adaptation, heterogeneity alignment, meta-learning, and knowledge distillation. For the first time, it characterizes the inherent trade-offs between performance and efficiency across heterogeneous mitigation strategies and identifies critical deployment barriers. The study constructs a structured panoramic map of OFL methodologies, clarifying promising directions for technical integration. Empirical validation demonstrates that single-round collaborative modeling reduces communication overhead by over 90% compared to conventional federated learning—providing theoretically grounded, practically viable solutions for privacy-sensitive, low-bandwidth applications, such as lightweight collaborative training of large language models.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

One-shot Federated Learning (OFL) is a distributed machine learning paradigm that constrains client-server communication to a single round, addressing privacy and communication overhead issues associated with multiple rounds of data exchange in traditional Federated Learning (FL). OFL demonstrates the practical potential for integration with future approaches that require collaborative training models, such as large language models (LLMs). However, current OFL methods face two major challenges: data heterogeneity and model heterogeneity, which result in subpar performance compared to conventional FL methods. Worse still, despite numerous studies addressing these limitations, a comprehensive summary is still lacking. To address these gaps, this paper presents a systematic analysis of the challenges faced by OFL and thoroughly reviews the current methods. We also offer an innovative categorization method and analyze the trade-offs of various techniques. Additionally, we discuss the most promising future directions and the technologies that should be integrated into the OFL field. This work aims to provide guidance and insights for future research.

Problem

• Addresses privacy in federated learning
• Reduces communication overhead in distributed systems
• Improves performance in heterogeneous data models

Innovation

• One-shot Federated Learning paradigm
• Addresses privacy, communication overhead
• Systematic analysis, categorization method