Federated Learning for Large Models in Medical Imaging: A Comprehensive Review

2025-08-28

AI Summary

To address data silos in medical imaging AI caused by stringent privacy regulations, this paper proposes a full-stack federated learning framework spanning physics-informed image reconstruction to clinical diagnosis. The framework innovatively enables joint optimization of upstream reconstruction and downstream clinical tasks, integrating physics-guided reconstruction, heterogeneous data alignment, low-communication distributed fine-tuning, and secure parameter aggregation. Evaluated on multi-task benchmarks—including CT/MRI reconstruction, tumor segmentation, and diagnostic classification—our method significantly improves cross-institutional model generalization while ensuring data remain localized. It achieves a favorable trade-off among accuracy, computational efficiency, and privacy preservation. This work establishes a scalable theoretical paradigm and systematic implementation for collaborative training and continual updating of large medical AI models in privacy-sensitive settings.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Artificial intelligence (AI) has demonstrated considerable potential in the realm of medical imaging. However, the development of high-performance AI models typically necessitates training on large-scale, centralized datasets. This approach is confronted with significant challenges due to strict patient privacy regulations and legal restrictions on data sharing and utilization. These limitations hinder the development of large-scale models in medical domains and impede continuous updates and training with new data. Federated Learning (FL), a privacy-preserving distributed training framework, offers a new solution by enabling collaborative model development across fragmented medical datasets. In this survey, we review FL's contributions at two stages of the full-stack medical analysis pipeline. First, in upstream tasks such as CT or MRI reconstruction, FL enables joint training of robust reconstruction networks on diverse, multi-institutional datasets, alleviating data scarcity while preserving confidentiality. Second, in downstream clinical tasks like tumor diagnosis and segmentation, FL supports continuous model updating by allowing local fine-tuning on new data without centralizing sensitive images. We comprehensively analyze FL implementations across the medical imaging pipeline, from physics-informed reconstruction networks to diagnostic AI systems, highlighting innovations that improve communication efficiency, align heterogeneous data, and ensure secure parameter aggregation. Meanwhile, this paper provides an outlook on future research directions, aiming to serve as a valuable reference for the field's development.

Problem

• Addressing data privacy in medical AI model training
• Enabling collaborative learning across fragmented medical datasets
• Supporting continuous model updates without centralized data sharing

Innovation

• Federated Learning for privacy-preserving distributed training
• Enables joint training on multi-institutional datasets
• Supports continuous model updating without centralizing data