FedSPC: Shared Parameter Correction for Personalized Federated Learning

2026-06-11

AI Summary

This work addresses the challenge in personalized federated learning (PFL) where divergent client-specific objectives lead to conflicting updates of shared parameters, thereby degrading the quality of the shared representation. To mitigate this issue, the authors propose FedSPC, a lightweight, plug-and-play modular correction method that applies control-variable adjustments exclusively to shared parameters while leaving personalized components untouched. FedSPC is compatible with a wide range of PFL architectures, including those with shared feature extractors, shared classifiers, and fully shared models enhanced with local regularization. Evaluated on CIFAR-100 and Tiny-ImageNet using ViT, ResNet-34, and VGG-11 backbones, FedSPC consistently improves the performance of diverse PFL methods—such as FedPer, FedRep, FedBABU, LG-FedAvg, and Ditto—by effectively alleviating optimization conflicts across clients and enhancing the robustness and expressiveness of the shared representation.

Abstract

Personalized federated learning (PFL) is one of the important approaches in federated learning for addressing statistical heterogeneity while enabling client-specific adaptation. Many PFL methods split the model into shared and personalized parameters, which are jointly trained on each client. However, this creates an optimization issue: shared parameters are updated by clients optimizing different local objectives, which can lead to inconsistent shared updates and weaken the shared representation. To address this problem, we propose Federated Shared Parameter Correction (FedSPC), a modular correction method for PFL. FedSPC applies control-variate correction only to the shared parameters of a given PFL method, while leaving personalized parameters unchanged. It can be integrated into three common PFL settings: shared feature extractors, shared classifiers, and fully shared models with local regularization. Experiments on CIFAR-100 and Tiny-ImageNet with ViT, ResNet-34, and VGG-11 show that FedSPC improves performance across representative PFL methods, including FedPer, FedRep, FedBABU, LG-FedAvg, and Ditto.

Problem

• Personalized Federated Learning
• Statistical Heterogeneity
• Shared Parameters
• Optimization Inconsistency
• Client-specific Adaptation

Innovation

• Personalized Federated Learning
• Shared Parameter Correction
• Control Variate
• Statistical Heterogeneity
• Modular Integration