Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation

2026-09-15

AI Summary

本文针对统计异质性限制联邦学习的问题，提出pFedKDH方法，通过全局知识蒸馏和本地头部适应来提高个性化模型的准确性。

Abstract

Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67\% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.

Problem

• statistical heterogeneity
• federated learning
• client-specific label distributions

Innovation

• Personalized Federated Learning
• Global Knowledge Distillation
• Local Head Adaptation
• Statistical Heterogeneity