FedQuad: Federated Stochastic Quadruplet Learning to Mitigate Data Heterogeneity

2025-09-04

AI Summary

In federated learning, data heterogeneity—particularly under few-shot and class-imbalanced settings—severely degrades the generalization capability of the global model. To address this, we propose FedQuad, the first framework to integrate stochastic quadruplet learning into federated learning. FedQuad explicitly optimizes intra-class compactness and inter-class separability across client-specific feature spaces, thereby decoupling client representations and enhancing robustness to non-IID data. Our method synergistically combines metric-learning-based quadruplet loss with federated averaging, jointly minimizing intra-class variance and maximizing inter-class variance during distributed training. Extensive experiments on CIFAR-10 and CIFAR-100 under diverse non-IID partitions demonstrate that FedQuad consistently outperforms state-of-the-art baselines, achieving particularly notable gains in low-data-volume and long-tailed distribution scenarios.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated Learning (FL) provides decentralised model training, which effectively tackles problems such as distributed data and privacy preservation. However, the generalisation of global models frequently faces challenges from data heterogeneity among clients. This challenge becomes even more pronounced when datasets are limited in size and class imbalance. To address data heterogeneity, we propose a novel method, extit{FedQuad}, that explicitly optimises smaller intra-class variance and larger inter-class variance across clients, thereby decreasing the negative impact of model aggregation on the global model over client representations. Our approach minimises the distance between similar pairs while maximising the distance between negative pairs, effectively disentangling client data in the shared feature space. We evaluate our method on the CIFAR-10 and CIFAR-100 datasets under various data distributions and with many clients, demonstrating superior performance compared to existing approaches. Furthermore, we provide a detailed analysis of metric learning-based strategies within both supervised and federated learning paradigms, highlighting their efficacy in addressing representational learning challenges in federated settings.

Problem

• Mitigating data heterogeneity in federated learning
• Reducing intra-class variance across clients
• Maximizing inter-class variance for better generalization

Innovation

• Optimizes smaller intra-class and larger inter-class variance
• Minimizes similar pairs distance while maximizing negative pairs distance
• Disentangles client data in shared feature space effectively