FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling

2026-04-17

AI Summary

This work addresses the degradation in model accuracy in federated learning caused by data heterogeneity and resource-constrained devices by proposing a personalization method based on element-wise parameter decoupling. Building upon an enhanced formulation of the Optimal Brain Damage theory, the approach leverages a first-order Taylor expansion of the federated loss to efficiently estimate each parameter’s sensitivity to local objectives at the server. A quantile-based thresholding mechanism is then employed to identify and retain only the most critical parameters for personalization. Notably, this study pioneers the integration of saliency-based pruning into federated parameter decoupling, enabling lightweight personalization under cloud-edge collaboration. Experimental results across diverse datasets and heterogeneous settings demonstrate that the method significantly outperforms existing approaches with minimal parameter adaptation.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Federated Learning (FL) faces challenges from client data heterogeneity and resource-constrained mobile devices, which can degrade model accuracy. Personalized Federated Learning (PFL) addresses this issue by adapting shared global knowledge to local data distributions. A promising approach in PFL is model decoupling, which separates the model into global and personalized parameters, raising the key question of which parameters should be personalized to balance global knowledge sharing and local adaptation. In this paper, we propose a Federated Optimal Brain Personalization (FedOBP) algorithm with a quantile-based thresholding mechanism and introduce an element-wise importance score. This score extends Optimal Brain Damage (OBD) pruning theory by incorporating a federated approximation of the first-order derivative in the Taylor expansion to evaluate the importance of each parameter for personalization. Moreover, we move the metric computation originally performed on clients to the server side, to alleviate the burden on resource-constrained mobile devices. To the best of our knowledge, this is the first work to bridge classical saliency-based pruning theory with federated parameter decoupling, providing a rigorous theoretical justification for selecting personalized parameters based on their sensitivity to local loss landscapes. Extensive experiments demonstrate that FedOBP outperforms state-of-the-art methods across diverse datasets and heterogeneity scenarios, while requiring personalization of only a very small number of personalized parameters.

Problem

• Federated Learning
• Personalized Federated Learning
• Model Decoupling
• Parameter Personalization
• Data Heterogeneity

Innovation

• Federated Learning
• Personalized Federated Learning
• Model Decoupling
• Optimal Brain Damage
• Element-wise Importance