Hierarchical Split Federated Learning: Convergence Analysis and System Optimization

2024-12-10

arXiv.org

AI Summary

Deploying large-scale AI models via federated learning (FL) on resource-constrained edge devices remains challenging; existing split FL (SFL) approaches are largely confined to two-tier architectures and fail to exploit heterogeneous multi-tier cloud–edge–device resources. Method: We propose Hierarchical Split Federated Learning (HSFL), the first framework supporting flexible, heterogeneous multi-level system deployments. Theoretically, we derive the first convergence upper bound for HSFL. Methodologically, we jointly optimize model partitioning locations and multi-tier aggregation strategies, and design a decomposable iterative descent algorithm to achieve Pareto-optimal trade-offs among accuracy, communication, and computation overhead. Results: Extensive simulations demonstrate that HSFL significantly improves convergence speed and model accuracy across arbitrary multi-tier topologies, while achieving superior communication and computational resource utilization compared to state-of-the-art SFL methods.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Multiagent Learning
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

As AI models expand in size, it has become increasingly challenging to deploy federated learning (FL) on resource-constrained edge devices. To tackle this issue, split federated learning (SFL) has emerged as an FL framework with reduced workload on edge devices via model splitting; it has received extensive attention from the research community in recent years. Nevertheless, most prior works on SFL focus only on a two-tier architecture without harnessing multi-tier cloudedge computing resources. In this paper, we intend to analyze and optimize the learning performance of SFL under multi-tier systems. Specifically, we propose the hierarchical SFL (HSFL) framework and derive its convergence bound. Based on the theoretical results, we formulate a joint optimization problem for model splitting (MS) and model aggregation (MA). To solve this rather hard problem, we then decompose it into MS and MA subproblems that can be solved via an iterative descending algorithm. Simulation results demonstrate that the tailored algorithm can effectively optimize MS and MA for SFL within virtually any multi-tier system.

Problem

• Optimize federated learning for resource-constrained edge devices
• Extend split federated learning to multi-tier cloud-edge systems
• Jointly optimize model splitting and aggregation in hierarchical SFL

Innovation

• Hierarchical SFL framework for multi-tier systems
• Joint optimization of model splitting and aggregation
• Iterative descending algorithm for solving subproblems