H-FedSN: Personalized Sparse Networks for Efficient and Accurate Hierarchical Federated Learning for IoT Applications

2024-12-09

AI Summary

To address high communication overhead, severe data heterogeneity, and imbalanced device distribution in hierarchical federated learning (HFL) for multi-tier IoT systems, this paper proposes a lightweight and efficient HFL framework. Methodologically, it introduces (1) a novel binary-mask-driven sparse shared/personalized layer architecture that significantly compresses model uploads, and (2) a two-tier edge–cloud Bayesian aggregation mechanism that dynamically weights non-IID client contributions using cumulative Beta-distribution-based confidence estimation. Extensive experiments on three real-world IoT datasets and MNIST demonstrate that the framework reduces communication costs by 58×–238× while maintaining state-of-the-art model accuracy. These results substantially enhance the practicality and deployability of HFL in resource-constrained IoT environments.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Humans and AI: Human-in-the-loop Machine Learning
• Multiagent Systems: Multiagent Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The proliferation of Internet of Things (IoT) has increased interest in federated learning (FL) for privacy-preserving distributed data utilization. However, traditional two-tier FL architectures inadequately adapt to multi-tier IoT environments. While Hierarchical Federated Learning (HFL) improves practicality in multi-tier IoT environments by multi-layer aggregation, it still faces challenges in communication efficiency and accuracy due to high data transfer volumes, data heterogeneity, and imbalanced device distribution, struggling to meet the low-latency and high-accuracy model training requirements of practical IoT scenarios. To overcome these limitations, we propose H-FedSN, an innovative approach for practical IoT environments. H-FedSN introduces a binary mask mechanism with shared and personalized layers to reduce communication overhead by creating a sparse network while keeping original weights frozen. To address data heterogeneity and imbalanced device distribution, we integrate personalized layers for local data adaptation and apply Bayesian aggregation with cumulative Beta distribution updates at edge and cloud levels, effectively balancing contributions from diverse client groups. Evaluations on three real-world IoT datasets and MNIST under non-IID settings demonstrate that H-FedSN significantly reduces communication costs by 58 to 238 times compared to HierFAVG while achieving high accuracy, making it highly effective for practical IoT applications in hierarchical federated learning scenarios.

Problem

• Hierarchical Federated Learning
• Internet of Things (IoT)
• Data Transmission Efficiency

Innovation

• Hierarchical Federated Learning
• Communication Efficiency
• Privacy Preservation