FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments

2026-02-26

AI Summary

This work addresses the performance degradation of federated learning under client data heterogeneity. Existing clustering-based approaches rely solely on either data or gradient similarity and restrict knowledge sharing across clusters, limiting their effectiveness. To overcome these limitations, we propose FedDAG, a novel framework that first integrates data distribution and gradient information to construct a weighted class-level similarity metric for more accurate client clustering. Furthermore, FedDAG introduces a dual-encoder architecture that enables both cluster-specific model specialization and cross-cluster feature transfer, while a cross-cluster gradient aggregation mechanism breaks the rigid isolation imposed by traditional clustering methods. Extensive experiments demonstrate that FedDAG consistently outperforms state-of-the-art clustered federated learning approaches across diverse heterogeneity settings and benchmark datasets.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Federated Learning (FL) enables a group of clients to collaboratively train a model without sharing individual data, but its performance drops when client data are heterogeneous. Clustered FL tackles this by grouping similar clients. However, existing clustered FL approaches rely solely on either data similarity or gradient similarity; however, this results in an incomplete assessment of client similarities. Prior clustered FL approaches also restrict knowledge and representation sharing to clients within the same cluster. This prevents cluster models from benefiting from the diverse client population across clusters. To address these limitations, FedDAG introduces a clustered FL framework, FedDAG, that employs a weighted, class-wise similarity metric that integrates both data and gradient information, providing a more holistic measure of similarity during clustering. In addition, FedDAG adopts a dual-encoder architecture for cluster models, comprising a primary encoder trained on its own clients'data and a secondary encoder refined using gradients from complementary clusters. This enables cross-cluster feature transfer while preserving cluster-specific specialization. Experiments on diverse benchmarks and data heterogeneity settings show that FedDAG consistently outperforms state-of-the-art clustered FL baselines in accuracy.

Problem

• Federated Learning
• Data Heterogeneity
• Clustered FL
• Client Similarity
• Cross-cluster Knowledge Sharing

Innovation

• clustered federated learning
• data-gradient integration
• cross-cluster knowledge transfer
• dual-encoder architecture
• heterogeneous environments