A Hierarchical Federated Learning Approach for the Internet of Things

2024-03-03

AI Summary

To address slow convergence, low accuracy, and high communication overhead in federated learning (FL) for large-scale IoT—caused by wide geographical distribution, scarce communication resources, and highly heterogeneous data—this paper proposes QHetFed, a hierarchical FL framework. Methodologically, QHetFed introduces: (1) a novel cooperative mechanism integrating intra-cluster gradient aggregation with inter-cluster model aggregation; (2) closed-form optimal solutions for critical hyperparameters (e.g., learning rate) under joint communication and computation constraints; and (3) adaptive data quantization coupled with latency-aware optimization. Theoretical analysis establishes convergence rate bounds and quantifies the optimality gap. Experiments on heterogeneous data benchmarks demonstrate that QHetFed achieves, on average, a 12.7% higher test accuracy and 35% lower communication cost compared to state-of-the-art hierarchical FL methods, while significantly accelerating convergence.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper presents a novel federated learning solution, QHetFed, suitable for large-scale Internet of Things deployments, addressing the challenges of large geographic span, communication resource limitation, and data heterogeneity. QHetFed is based on hierarchical federated learning over multiple device sets, where the learning process and learning parameters take the necessary data quantization and the data heterogeneity into consideration to achieve high accuracy and fast convergence. Unlike conventional hierarchical federated learning algorithms, the proposed approach combines gradient aggregation in intra-set iterations with model aggregation in inter-set iterations. We offer a comprehensive analytical framework to evaluate its optimality gap and convergence rate, and give a closed form expression for the optimal learning parameters under a deadline, that accounts for communication and computation times. Our findings reveal that QHetFed consistently achieves high learning accuracy and significantly outperforms other hierarchical algorithms, particularly in scenarios with heterogeneous data distributions.

Problem

• Addresses IoT challenges: large span, resource limits, data heterogeneity
• Proposes QHetFed for high accuracy, fast convergence in federated learning
• Optimizes learning parameters under communication and computation deadlines

Innovation

• Hierarchical federated learning for IoT
• Quantization and heterogeneity-aware parameters
• Gradient and model aggregation combination