Exploiting Label Skewness for Spiking Neural Networks in Federated Learning

2024-12-23

AI Summary

To address label non-IID-induced local model drift and degraded global performance in spiking neural networks (SNNs) under federated learning, this paper proposes FedLEC—the first SNN-specific federated learning framework. FedLEC innovatively integrates intra-client label-distribution-aware weight calibration with inter-client knowledge distillation to systematically mitigate bias accumulation under label skew. Designed for resource-constrained edge devices and strict data privacy requirements, it natively supports brain-inspired neuromorphic data. Extensive experiments across five benchmark datasets—including neuromorphic benchmarks—demonstrate that FedLEC consistently outperforms eight state-of-the-art federated learning methods: the global SNN achieves an average accuracy improvement of 11.59%, alongside significantly enhanced generalization and robustness against label heterogeneity.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

The energy efficiency of deep spiking neural networks (SNNs) aligns with the constraints of resource-limited edge devices, positioning SNNs as a promising foundation for intelligent applications leveraging the extensive data collected by these devices. To address data privacy concerns when deploying SNNs on edge devices, federated learning (FL) facilitates collaborative model training by leveraging data distributed across edge devices without transmitting local data to a central server. However, existing FL approaches struggle with label-skewed data across devices, which leads to drift in local SNN models and degrades the performance of the global SNN model. In this paper, we propose a novel framework called FedLEC, which incorporates intra-client label weight calibration to balance the learning intensity across local labels and inter-client knowledge distillation to mitigate local SNN model bias caused by label absence. Extensive experiments with three different structured SNNs across five datasets (i.e., three non-neuromorphic and two neuromorphic datasets) demonstrate the efficiency of FedLEC. Compared to eight state-of-the-art FL algorithms, FedLEC achieves an average accuracy improvement of approximately 11.59% for the global SNN model under various label skew distribution settings.

Problem

• Federated Learning
• Imbalanced Data Distribution
• Spike Neural Networks

Innovation

• FedLEC
• Imbalanced Labels
• Knowledge Sharing