RCC-PFL: Robust Client Clustering under Noisy Labels in Personalized Federated Learning

2025-03-25

AI Summary

In personalized federated learning (PFL), noisy labels impair accurate estimation of user cluster identities. To address this, we propose a label-agnostic, single-pass unsupervised clustering method that eliminates reliance on labels entirely. Our approach performs client-local feature extraction and employs a robust similarity metric in the feature space to conduct clustering once—during a pretraining phase—bypassing iterative optimization. The core contribution is the first label-agnostic pretraining clustering mechanism, which substantially reduces communication rounds and computational overhead. Extensive experiments across multiple models and benchmark datasets demonstrate that our method achieves significantly higher average clustering accuracy and markedly lower variance compared to mainstream iterative baselines, establishing comprehensive superiority in both effectiveness and efficiency.

Technology Category

• Machine Learning: Clustering
• Humans and AI: Human-in-the-loop Machine Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We address the problem of cluster identity estimation in a personalized federated learning (PFL) setting in which users aim to learn different personal models. The backbone of effective learning in such a setting is to cluster users into groups whose objectives are similar. A typical approach in the literature is to achieve this by training users' data on different proposed personal models and assign them to groups based on which model achieves the lowest value of the users' loss functions. This process is to be done iteratively until group identities converge. A key challenge in such a setting arises when users have noisy labeled data, which may produce misleading values of their loss functions, and hence lead to ineffective clustering. To overcome this challenge, we propose a label-agnostic data similarity-based clustering algorithm, coined RCC-PFL, with three main advantages: the cluster identity estimation procedure is independent from the training labels; it is a one-shot clustering algorithm performed prior to the training; and it requires fewer communication rounds and less computation compared to iterative-based clustering methods. We validate our proposed algorithm using various models and datasets and show that it outperforms multiple baselines in terms of average accuracy and variance reduction.

Problem

• Estimating cluster identity in personalized federated learning with noisy labels
• Improving clustering accuracy by reducing label dependency
• Reducing communication rounds and computation in client clustering

Innovation

• Label-agnostic data similarity-based clustering
• One-shot clustering before training
• Fewer communication rounds and computation