SAM-Fed: SAM-Guided Federated Semi-Supervised Learning for Medical Image Segmentation

2025-11-18

AI Summary

Medical image segmentation is hindered by data privacy constraints and scarcity of high-quality annotations. While federated semi-supervised learning (FSSL) offers promise, it suffers from low-quality pseudo-labels and performance degradation due to client model heterogeneity and lightweight architectures. To address these challenges, we propose SAM-Fed—the first FSSL framework integrating the Segment Anything Model (SAM) as a server-side teacher. It enables pixel-level knowledge transfer to lightweight, heterogeneous client models via dual knowledge distillation and an adaptive consistency mechanism. Crucially, SAM-Fed preserves data privacy while significantly improving pseudo-label reliability and segmentation robustness. Extensive experiments on skin lesion and polyp segmentation demonstrate that SAM-Fed consistently outperforms state-of-the-art methods under both homogeneous and heterogeneous federated settings, achieving average Dice score improvements of 2.1–3.8 percentage points.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Segmentation
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Medical image segmentation is clinically important, yet data privacy and the cost of expert annotation limit the availability of labeled data. Federated semi-supervised learning (FSSL) offers a solution but faces two challenges: pseudo-label reliability depends on the strength of local models, and client devices often require compact or heterogeneous architectures due to limited computational resources. These constraints reduce the quality and stability of pseudo-labels, while large models, though more accurate, cannot be trained or used for routine inference on client devices. We propose SAM-Fed, a federated semi-supervised framework that leverages a high-capacity segmentation foundation model to guide lightweight clients during training. SAM-Fed combines dual knowledge distillation with an adaptive agreement mechanism to refine pixel-level supervision. Experiments on skin lesion and polyp segmentation across homogeneous and heterogeneous settings show that SAM-Fed consistently outperforms state-of-the-art FSSL methods.

Problem

• Improving pseudo-label reliability in federated semi-supervised medical image segmentation
• Enabling lightweight client models to leverage high-capacity foundation models
• Addressing computational constraints in heterogeneous federated learning environments

Innovation

• Leverages segmentation foundation model for lightweight clients
• Combines dual knowledge distillation with adaptive agreement
• Refines pixel-level supervision for medical image segmentation