DualFed: Enjoying both Generalization and Personalization in Federated Learning via Hierachical Representations

2024-07-25

ACM Multimedia

AI Summary

In personalized federated learning (PFL), model generalizability and client-specific personalization performance are often mutually constraining. To address this, we propose the Dual-Rep Architecture—a unified framework that jointly learns generalized and personalized representations in a decoupled manner for the first time. Our method adopts a three-tier structure comprising an encoder, a lightweight projection network, and a classifier; it enables collaborative training via client-side fine-tuning and server-side global aggregation, eliminating the computational overhead of multi-layer representation search. Extensive experiments on multiple benchmark datasets demonstrate that our approach significantly outperforms existing state-of-the-art PFL methods in both generalization accuracy (on unseen clients) and personalized accuracy (on local data). Notably, it achieves the first empirically validated, simultaneous improvement in generalizability and personalization capability within the PFL paradigm.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In personalized federated learning (PFL), it is widely recognized that achieving both high model generalization and effective personalization poses a significant challenge due to their conflicting nature. As a result, existing PFL methods can only manage a trade-off between these two objectives. This raises an interesting question: Is it feasible to develop a model capable of achieving both objectives simultaneously? Our paper presents an affirmative answer, and the key lies in the observation that deep models inherently exhibit hierarchical architectures, which produce representations with various levels of generalization and personalization at different stages. A straightforward approach stemming from this observation is to select multiple representations from these layers and combine them to concurrently achieve generalization and personalization. However, the number of candidate representations is commonly huge, which makes this method infeasible due to high computational costs.To address this problem, we propose DualFed, a new method that can directly yield dual representations correspond to generalization and personalization respectively, thereby simplifying the optimization task. Specifically, DualFed inserts a personalized projection network between the encoder and classifier. The pre-projection representations are able to capture generalized information shareable across clients, and the post-projection representations are effective to capture task-specific information on local clients. This design minimizes the mutual interference between generalization and personalization, thereby achieving a win-win situation. Extensive experiments show that DualFed can outperform other FL methods. Code is available at https://github.com/GuogangZhu/DualFed.

Problem

• Achieve generalization and personalization simultaneously in federated learning.
• Propose DualFed to simplify optimization of dual representations.
• Minimize interference between shared and task-specific information.

Innovation

• Hierarchical representations in PFL
• Personalized projection network insertion
• Dual representations for generalization and personalization