FedKDX: Federated Learning with Negative Knowledge Distillation for Enhanced Healthcare AI Systems

2026-01-08

arXiv.org

AI Summary

This work addresses the limited generalization of medical federated learning under non-IID data and its reliance solely on positive knowledge transfer. To overcome these limitations, we propose FedKDX, a novel framework that, for the first time, incorporates negative knowledge distillation by synergistically integrating knowledge distillation, contrastive learning, and federated learning. This approach enhances model robustness by leveraging both target and non-target information while preserving data privacy. FedKDX significantly improves generalization performance on statistically heterogeneous medical data and reduces communication overhead. Experimental results on benchmark datasets—including SLEEP, UCI-HAR, and PAMAP2—demonstrate that FedKDX achieves up to a 2.53% accuracy gain over existing methods in non-IID settings and exhibits faster convergence.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper introduces FedKDX, a federated learning framework that addresses limitations in healthcare AI through Negative Knowledge Distillation (NKD). Unlike existing approaches that focus solely on positive knowledge transfer, FedKDX captures both target and non-target information to improve model generalization in healthcare applications. The framework integrates multiple knowledge transfer techniques--including traditional knowledge distillation, contrastive learning, and NKD--within a unified architecture that maintains privacy while reducing communication costs. Through experiments on healthcare datasets (SLEEP, UCI-HAR, and PAMAP2), FedKDX demonstrates improved accuracy (up to 2.53% over state-of-the-art methods), faster convergence, and better performance on non-IID data distributions. Theoretical analysis supports NKD's contribution to addressing statistical heterogeneity in distributed healthcare data. The approach shows promise for privacy-sensitive medical applications under regulatory frameworks like HIPAA and GDPR, offering a balanced solution between performance and practical implementation requirements in decentralized healthcare settings. The code and model are available at https://github.com/phamdinhdat-ai/Fed_2024.

Problem

• Federated Learning
• Healthcare AI
• Statistical Heterogeneity
• Non-IID Data
• Privacy-Preserving

Innovation

• Negative Knowledge Distillation
• Federated Learning
• Healthcare AI
• Non-IID Data
• Privacy-Preserving Machine Learning