FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis

2024-12-24

AI Summary

To address client drift and high communication overhead caused by cross-hospital non-IID data in medical federated learning, this paper proposes a model-guided knowledge filtering and compression framework. Methodologically, it integrates implicit distribution constraints to enhance local knowledge quality, designs relation-aware supervised contrastive learning to improve global model consistency, and introduces adaptive knowledge selection coupled with lightweight synthetic data generation. Its key innovation lies in the first joint modeling of model-guided knowledge distillation and distribution-aware contrastive learning, effectively mitigating non-IID bias. Evaluated on multiple medical imaging tasks, the method reduces communication rounds by over 30% compared to state-of-the-art approaches while improving average accuracy by 2.1–4.7 percentage points and significantly enhancing robustness.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Federated learning has become a promising solution for collaboration among medical institutions. However, data owned by each institution would be highly heterogeneous and the distribution is always non-independent and identical distribution (non-IID), resulting in client drift and unsatisfactory performance. Despite existing federated learning methods attempting to solve the non-IID problems, they still show marginal advantages but rely on frequent communication which would incur high costs and privacy concerns. In this paper, we propose a novel federated learning method: extbf{Fed}erated learning via extbf{V}aluable extbf{C}ondensed extbf{K}nowledge (FedVCK). We enhance the quality of condensed knowledge and select the most necessary knowledge guided by models, to tackle the non-IID problem within limited communication budgets effectively. Specifically, on the client side, we condense the knowledge of each client into a small dataset and further enhance the condensation procedure with latent distribution constraints, facilitating the effective capture of high-quality knowledge. During each round, we specifically target and condense knowledge that has not been assimilated by the current model, thereby preventing unnecessary repetition of homogeneous knowledge and minimizing the frequency of communications required. On the server side, we propose relational supervised contrastive learning to provide more supervision signals to aid the global model updating. Comprehensive experiments across various medical tasks show that FedVCK can outperform state-of-the-art methods, demonstrating that it's non-IID robust and communication-efficient.

Problem

• Federated Learning
• Data Heterogeneity
• Communication Efficiency

Innovation

• FedVCK
• Efficient Federated Learning
• Medical Image Analysis