GC-Fed: Gradient Centralized Federated Learning with Partial Client Participation

2025-03-17

AI Summary

In federated learning under high data heterogeneity, severe client drift occurs, and conventional reference-model-based gradient correction methods suffer from instability in partial participation settings. Method: This paper proposes a reference-model-free hierarchical gradient centralization mechanism. It performs coordinated gradient correction on the feature extraction and classifier layers separately during local training and global aggregation, establishing a local-global two-stage hybrid optimization paradigm. Contribution/Results: We provide theoretical convergence guarantees showing strict improvement over standard FedAvg. Empirical evaluation under strong heterogeneity demonstrates up to a 20% absolute improvement in test accuracy over baseline methods, significantly outperforming existing approaches and achieving state-of-the-art performance.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multi-source information fusion (MSIF) leverages diverse data streams to enhance decision-making, situational awareness, and system resilience. Federated Learning (FL) enables MSIF while preserving privacy but suffers from client drift under high data heterogeneity, leading to performance degradation. Traditional mitigation strategies rely on reference-based gradient adjustments, which can be unstable in partial participation settings. To address this, we propose Gradient Centralized Federated Learning (GC-Fed), a reference-free gradient correction method inspired by Gradient Centralization (GC). We introduce Local GC and Global GC, applying GC during local training and global aggregation, respectively. Our hybrid GC-Fed approach selectively applies GC at the feature extraction layer locally and at the classifier layer globally, improving training stability and model performance. Theoretical analysis and empirical results demonstrate that GC-Fed mitigates client drift and achieves state-of-the-art accuracy gains of up to 20% in heterogeneous settings.

Problem

• Mitigates client drift in Federated Learning under data heterogeneity.
• Proposes Gradient Centralized Federated Learning (GC-Fed) for stability.
• Improves model accuracy by up to 20% in heterogeneous settings.

Innovation

• Gradient Centralized Federated Learning (GC-Fed)
• Local and Global Gradient Centralization (GC)
• Hybrid GC-Fed for improved stability and accuracy