FedBiCross: A Bi-Level Optimization Framework to Tackle Non-IID Challenges in Data-Free One-Shot Federated Learning on Medical Data

2026-01-05

arXiv.org

AI Summary

This work addresses the challenge of low-quality soft labels and poor supervision in existing data-free, one-shot federated learning methods under non-IID data settings, which stem from naive averaging of global teacher models. To overcome this limitation, the authors propose FedBiCross, a novel framework that achieves effective personalization within a single communication round through a three-stage pipeline: client clustering based on output similarity, bi-level cross-cluster optimization, and personalized knowledge distillation. FedBiCross introduces, for the first time, a dynamically weighted cross-cluster knowledge fusion mechanism that mitigates negative transfer and enhances model personalization. Extensive experiments demonstrate that FedBiCross significantly outperforms state-of-the-art methods across four medical imaging datasets and consistently delivers robust performance gains under varying degrees of data heterogeneity.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Data-free knowledge distillation-based one-shot federated learning (OSFL) trains a model in a single communication round without sharing raw data, making OSFL attractive for privacy-sensitive medical applications. However, existing methods aggregate predictions from all clients to form a global teacher. Under non-IID data, conflicting predictions cancel out during averaging, yielding near-uniform soft labels that provide weak supervision for distillation. We propose FedBiCross, a personalized OSFL framework with three stages: (1) clustering clients by model output similarity to form coherent sub-ensembles, (2) bi-level cross-cluster optimization that learns adaptive weights to selectively leverage beneficial cross-cluster knowledge while suppressing negative transfer, and (3) personalized distillation for client-specific adaptation. Experiments on four medical image datasets demonstrate that FedBiCross consistently outperforms state-of-the-art baselines across different non-IID degrees.

Problem

• non-IID
• data-free
• one-shot federated learning
• medical data
• knowledge distillation

Innovation

• bi-level optimization
• client clustering
• cross-cluster knowledge transfer
• personalized distillation
• non-IID federated learning