Federated Multi-Task Clustering

2025-12-28

AI Summary

To address three key challenges in decentralized federated clustering—(i) the difficulty of distributing traditional spectral clustering, (ii) poor generalization of existing methods due to reliance on unreliable pseudo-labels, and (iii) neglect of latent structural correlations across heterogeneous clients—this paper proposes the first federated multi-task clustering framework. The framework jointly designs client-specific nonlinear mappings and server-side tensor-based low-rank correlation modeling, enabling privacy-preserving co-optimization of shared structural discovery and personalized clustering via the Alternating Direction Method of Multipliers (ADMM). Extensive experiments on multiple real-world datasets demonstrate significant improvements over state-of-the-art federated clustering methods, achieving higher clustering accuracy and superior generalization—particularly in out-of-sample extrapolation—across heterogeneous clients.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

Spectral clustering has emerged as one of the most effective clustering algorithms due to its superior performance. However, most existing models are designed for centralized settings, rendering them inapplicable in modern decentralized environments. Moreover, current federated learning approaches often suffer from poor generalization performance due to reliance on unreliable pseudo-labels, and fail to capture the latent correlations amongst heterogeneous clients. To tackle these limitations, this paper proposes a novel framework named Federated Multi-Task Clustering (i.e.,FMTC), which intends to learn personalized clustering models for heterogeneous clients while collaboratively leveraging their shared underlying structure in a privacy-preserving manner. More specifically, the FMTC framework is composed of two main components: client-side personalized clustering module, which learns a parameterized mapping model to support robust out-of-sample inference, bypassing the need for unreliable pseudo-labels; and server-side tensorial correlation module, which explicitly captures the shared knowledge across all clients. This is achieved by organizing all client models into a unified tensor and applying a low-rank regularization to discover their common subspace. To solve this joint optimization problem, we derive an efficient, privacy-preserving distributed algorithm based on the Alternating Direction Method of Multipliers, which decomposes the global problem into parallel local updates on clients and an aggregation step on the server. To the end, several extensive experiments on multiple real-world datasets demonstrate that our proposed FMTC framework significantly outperforms various baseline and state-of-the-art federated clustering algorithms.

Problem

• Develops personalized clustering models for heterogeneous clients
• Captures shared knowledge across clients using tensor-based correlation
• Ensures privacy via distributed optimization without pseudo-labels

Innovation

• Federated multi-task clustering for personalized heterogeneous client models
• Client-side mapping bypasses unreliable pseudo-labels for robust inference
• Server-side tensorial correlation captures shared knowledge via low-rank regularization