Learning to Collaborate Over Graphs: A Selective Federated Multi-Task Learning Approach

2025-06-11

AI Summary

To address the coexistence of personalization modeling and negative transfer in federated multitask learning (FMTL) under heterogeneous clients, this paper proposes a dynamic collaborative graph-driven personalized FMTL framework. Methodologically: (i) it introduces a novel community-detection-based (Louvain) graph regularization mechanism that dynamically constructs homogeneous client subgroups according to task similarity; (ii) it designs a lightweight feature anchor module coupled with a task-adaptive classification head sharing strategy, enabling selective knowledge transfer while preserving local model characteristics. Experiments on two heterogeneous benchmark datasets demonstrate significant improvements over state-of-the-art methods: 42% reduction in communication overhead, 35% improvement in computational efficiency, 58% reduction in client performance variance, and concurrent enhancements in fairness and generalization.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Graph Mining, Social Network Analysis & Community
• Multiagent Systems: Mechanism Design

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present a novel federated multi-task learning method that leverages cross-client similarity to enable personalized learning for each client. To avoid transmitting the entire model to the parameter server, we propose a communication-efficient scheme that introduces a feature anchor, a compact vector representation that summarizes the features learned from the client's local classes. This feature anchor is shared with the server to account for local clients' distribution. In addition, the clients share the classification heads, a lightweight linear layer, and perform a graph-based regularization to enable collaboration among clients. By modeling collaboration between clients as a dynamic graph and continuously updating and refining this graph, we can account for any drift from the clients. To ensure beneficial knowledge transfer and prevent negative collaboration, we leverage a community detection-based approach that partitions this dynamic graph into homogeneous communities, maximizing the sum of task similarities, represented as the graph edges' weights, within each community. This mechanism restricts collaboration to highly similar clients within their formed communities, ensuring positive interaction and preserving personalization. Extensive experiments on two heterogeneous datasets demonstrate that our method significantly outperforms state-of-the-art baselines. Furthermore, we show that our method exhibits superior computation and communication efficiency and promotes fairness across clients.

Problem

• Enables personalized learning via cross-client similarity in federated multi-task learning.
• Proposes communication-efficient feature anchors to summarize local client distributions.
• Uses dynamic graph and community detection to ensure beneficial client collaboration.

Innovation

• Feature anchor for efficient communication
• Graph-based regularization for client collaboration
• Community detection for beneficial knowledge transfer