EMO: Edge Model Overlays to Scale Model Size in Federated Learning

2025-04-01

AI Summary

To address the challenge of training large-scale models via federated learning (FL) on resource-constrained edge devices, this paper proposes Augmented Federated Learning (AFL), a novel framework introducing an edge-end model stacking architecture. AFL enables both horizontal and vertical model scaling without modifying the standard FL workflow. It decouples computational dependencies through a hierarchical activation-based replay buffer, integrates convergence-aware communication control, and incorporates an ensemble inference module—collectively mitigating the tight data-flow coupling, high communication overhead, and computational bottlenecks inherent in split federated learning (SFL). Experiments demonstrate that AFL achieves up to 17.77% higher test accuracy than standard FL, reduces communication volume to 1/7.17 of SFL, and shortens training time to 1/6.9. This work establishes an efficient, scalable paradigm for large-model federated training in resource-limited edge environments.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Federated Learning (FL) trains machine learning models on edge devices with distributed data. However, the computational and memory limitations of these devices restrict the training of large models using FL. Split Federated Learning (SFL) addresses this challenge by distributing the model across the device and server, but it introduces a tightly coupled data flow, leading to computational bottlenecks and high communication costs. We propose EMO as a solution to enable the training of large models in FL while mitigating the challenges of SFL. EMO introduces Edge Model Overlay(s) between the device and server, enabling the creation of a larger ensemble model without modifying the FL workflow. The key innovation in EMO is Augmented Federated Learning (AFL), which builds an ensemble model by connecting the original (smaller) FL model with model(s) trained in the overlay(s) to facilitate horizontal or vertical scaling. This is accomplished through three key modules: a hierarchical activation replay cache to decouple AFL from FL, a convergence-aware communication controller to optimize communication overhead, and an ensemble inference module. Evaluations on a real-world prototype show that EMO improves accuracy by up to 17.77% compared to FL, and reduces communication costs by up to 7.17x and decreases training time by up to 6.9x compared to SFL.

Problem

• Overcoming device limitations in Federated Learning for large models
• Reducing communication costs in Split Federated Learning
• Enhancing model accuracy and training efficiency in FL

Innovation

• Edge Model Overlays enable large model training
• Augmented Federated Learning connects FL and overlay models
• Hierarchical cache and controller optimize communication