🤖 AI Summary
To address data and model heterogeneity in serverless decentralized federated learning—stemming from the absence of a global model—this paper proposes DeSA, a framework that enables collaborative learning of generalizable representations among clients via shared synthetic anchors, without centralized coordination. Methodologically, DeSA is the first to integrate domain adaptation theory with knowledge distillation in a decentralized setting, introducing two complementary loss terms: REG (latent-space distribution regularization) and KD (knowledge distillation), specifically designed to support decentralized graph neural network training. Extensive experiments demonstrate that DeSA significantly improves both cross-domain and in-domain accuracy across clients under diverse Non-IID data distributions. It also enhances convergence stability, reduces communication overhead, and achieves superior generalization performance compared to state-of-the-art decentralized FL approaches.
📝 Abstract
Conventional Federated Learning (FL) involves collaborative training of a global model while maintaining user data privacy. One of its branches, decentralized FL, is a serverless network that allows clients to own and optimize different local models separately, which results in saving management and communication resources. Despite the promising advancements in decentralized FL, it may reduce model generalizability due to lacking a global model. In this scenario, managing data and model heterogeneity among clients becomes a crucial problem, which poses a unique challenge that must be overcome: How can every client's local model learn generalizable representation in a decentralized manner? To address this challenge, we propose a novel Decentralized FL technique by introducing Synthetic Anchors, dubbed as DeSA. Based on the theory of domain adaptation and Knowledge Distillation (KD), we theoretically and empirically show that synthesizing global anchors based on raw data distribution facilitates mutual knowledge transfer. We further design two effective regularization terms for local training: 1) REG loss that regularizes the distribution of the client's latent embedding with the anchors and 2) KD loss that enables clients to learn from others. Through extensive experiments on diverse client data distributions, we showcase the effectiveness of DeSA in enhancing both inter- and intra-domain accuracy of each client.