Sharpness-Aware Minimization for Generalized Embedding Learning in Federated Recommendation

2026-03-11

AI Summary

This work addresses the challenge of highly heterogeneous and sparse client data in federated recommender systems, which impedes the stable learning of universal item embeddings and limits model generalization. To tackle this issue, the paper reframes the problem from an item-centric perspective and formulates it as a multi-task learning paradigm. It introduces Sharpness-Aware Minimization (SAM) into federated recommendation for the first time, proposing the FedRecGEL framework that enhances the stability and generalization of item embeddings by optimizing the flatness of the loss landscape. Extensive experiments on four benchmark datasets demonstrate that FedRecGEL significantly outperforms existing methods, effectively improving both model robustness and recommendation performance.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Recommender Systems
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Federated recommender systems enable collaborative model training while keeping user interaction data local and sharing only essential model parameters, thereby mitigating privacy risks. However, existing methods overlook a critical issue, i.e., the stable learning of a generalized item embedding throughout the federated recommender system training process. Item embedding plays a central role in facilitating knowledge sharing across clients. Yet, under the cross-device setting, local data distributions exhibit significant heterogeneity and sparsity, exacerbating the difficulty of learning generalized embeddings. These factors make the stable learning of generalized item embeddings both indispensable for effective federated recommendation and inherently difficult to achieve. To fill this gap, we propose a new federated recommendation framework, named Federated Recommendation with Generalized Embedding Learning (FedRecGEL). We reformulate the federated recommendation problem from an item-centered perspective and cast it as a multi-task learning problem, aiming to learn generalized embeddings throughout the training procedure. Based on theoretical analysis, we employ sharpness-aware minimization to address the generalization problem, thereby stabilizing the training process and enhancing recommendation performance. Extensive experiments on four datasets demonstrate the effectiveness of FedRecGEL in significantly improving federated recommendation performance. Our code is available at https://github.com/anonymifish/FedRecGEL.

Problem

• federated recommendation
• generalized embedding
• item embedding
• data heterogeneity
• embedding stability

Innovation

• Federated Recommendation
• Generalized Embedding Learning
• Sharpness-Aware Minimization
• Multi-task Learning
• Item-Centered Perspective