FedSCA: Federated Tuning with Similarity-guided Collaborative Aggregation for Heterogeneous Medical Image Segmentation

2025-03-19

AI Summary

To address the challenges of data silos across institutions, strict privacy constraints, and non-IID data distributions that impede fine-tuning foundation models for medical image segmentation, this paper proposes FedSCA, a federated fine-tuning framework. FedSCA introduces two key innovations: (1) a novel Similarity-Guided Collaborative Aggregation (SGCA) mechanism at the server, which performs weighted model aggregation based on inter-client model similarity; and (2) a communication-efficient strategy integrating Parameter-Efficient Fine-Tuning (PEFT) with sparse uploading of low-level adapter parameters, balancing communication overhead and representational capacity. Critically, FedSCA operates without sharing raw data, effectively mitigating non-IID bias and system heterogeneity. Evaluated on three mainstream benchmarks for federated medical image segmentation, FedSCA achieves new state-of-the-art performance, demonstrating significant improvements in generalization capability and convergence stability.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Segmentation
• Multiagent Systems: Other Foundations of Multi Agent Systems

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Transformer-based foundation models (FMs) have recently demonstrated remarkable performance in medical image segmentation. However, scaling these models is challenging due to the limited size of medical image datasets within isolated hospitals, where data centralization is restricted due to privacy concerns. These constraints, combined with the data-intensive nature of FMs, hinder their broader application. Integrating federated learning (FL) with foundation models (FLFM) fine-tuning offers a potential solution to these challenges by enabling collaborative model training without data sharing, thus allowing FMs to take advantage of a diverse pool of sensitive medical image data across hospitals/clients. However, non-independent and identically distributed (non-IID) data among clients, paired with computational and communication constraints in federated environments, presents an additional challenge that limits further performance improvements and remains inadequately addressed in existing studies. In this work, we propose a novel FLFM fine-tuning framework, underline{ extbf{Fed}}erated tuning with underline{ extbf{S}}imilarity-guided underline{ extbf{C}}ollaborative underline{ extbf{A}}ggregation (FedSCA), encompassing all phases of the FL process. This includes (1) specially designed parameter-efficient fine-tuning (PEFT) for local client training to enhance computational efficiency; (2) partial low-level adapter transmission for communication efficiency; and (3) similarity-guided collaborative aggregation (SGCA) on the server side to address non-IID issues. Extensive experiments on three FL benchmarks for medical image segmentation demonstrate the effectiveness of our proposed FedSCA, establishing new SOTA performance.

Problem

• Addresses limited medical image dataset sizes in isolated hospitals
• Solves non-IID data issues in federated learning environments
• Improves computational and communication efficiency in FLFM fine-tuning

Innovation

• Parameter-efficient fine-tuning for local training
• Partial low-level adapter transmission for efficiency
• Similarity-guided collaborative aggregation for non-IID