SHeRL-FL: When Representation Learning Meets Split Learning in Hierarchical Federated Learning

2025-08-11

AI Summary

To address high communication overhead, severe device computational heterogeneity, and complex model aggregation in large-scale federated learning (FL), this paper proposes a novel framework integrating representation learning with hierarchical split learning. The framework decouples representation optimization between clients and the server at intermediate network layers, enabling edge-only training without cloud involvement—thereby significantly reducing cross-layer coordination complexity and communication burden. Specifically, it embeds deep neural network intermediate-layer representation learning into a hierarchical split FL architecture. Extensive evaluations on AlexNet, ResNet, and U-Net demonstrate its effectiveness: compared to centralized FL and HierFL, communication volume is reduced by over 90%; relative to SplitFed, it decreases by 50%. Moreover, the framework achieves improved classification accuracy on CIFAR-10/100 and HAM10000, and enhanced segmentation performance on ISIC-2018, validating its efficacy across diverse tasks and models.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated learning (FL) is a promising approach for addressing scalability and latency issues in large-scale networks by enabling collaborative model training without requiring the sharing of raw data. However, existing FL frameworks often overlook the computational heterogeneity of edge clients and the growing training burden on resource-limited devices. However, FL suffers from high communication costs and complex model aggregation, especially with large models. Previous works combine split learning (SL) and hierarchical FL (HierFL) to reduce device-side computation and improve scalability, but this introduces training complexity due to coordination across tiers. To address these issues, we propose SHeRL-FL, which integrates SL and hierarchical model aggregation and incorporates representation learning at intermediate layers. By allowing clients and edge servers to compute training objectives independently of the cloud, SHeRL-FL significantly reduces both coordination complexity and communication overhead. To evaluate the effectiveness and efficiency of SHeRL-FL, we performed experiments on image classification tasks using CIFAR-10, CIFAR-100, and HAM10000 with AlexNet, ResNet-18, and ResNet-50 in both IID and non-IID settings. In addition, we evaluate performance on image segmentation tasks using the ISIC-2018 dataset with a ResNet-50-based U-Net. Experimental results demonstrate that SHeRL-FL reduces data transmission by over 90% compared to centralized FL and HierFL, and by 50% compared to SplitFed, which is a hybrid of FL and SL, and further improves hierarchical split learning methods.

Problem

• Reduces communication costs in hierarchical federated learning
• Addresses computational heterogeneity of edge clients
• Minimizes coordination complexity across network tiers

Innovation

• Integrates split learning and hierarchical model aggregation
• Uses representation learning at intermediate layers
• Reduces coordination complexity and communication overhead