SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation

2025-06-10

AI Summary

To address the high cost of high-quality pixel-level annotations and the abundance of unlabeled data in medical image segmentation, this paper proposes SSS, a semi-supervised framework. Methodologically: (1) it introduces Discriminative Feature Enhancement (DFE), a novel mechanism that strengthens inter-view feature discriminability; (2) it designs a Physics-Constrained Sliding-Window prompt generator (PCSW), enabling the first efficient adaptation of SAM-2 to semi-supervised medical segmentation by integrating domain-specific physical constraints with sliding-window prompting; and (3) it adopts a single-stream weak-strong consistency architecture coupled with multi-scale feature similarity/dissimilarity modeling. Evaluated on ACDC and BHSD benchmarks, SSS achieves state-of-the-art performance—e.g., a mean Dice score of 53.15 (+3.65) on BHSD. The implementation is publicly available.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance

Abstract

In the era of information explosion, efficiently leveraging large-scale unlabeled data while minimizing the reliance on high-quality pixel-level annotations remains a critical challenge in the field of medical imaging. Semi-supervised learning (SSL) enhances the utilization of unlabeled data by facilitating knowledge transfer, significantly improving the performance of fully supervised models and emerging as a highly promising research direction in medical image analysis. Inspired by the ability of Vision Foundation Models (e.g., SAM-2) to provide rich prior knowledge, we propose SSS (Semi-Supervised SAM-2), a novel approach that leverages SAM-2's robust feature extraction capabilities to uncover latent knowledge in unlabeled medical images, thus effectively enhancing feature support for fully supervised medical image segmentation. Specifically, building upon the single-stream"weak-to-strong"consistency regularization framework, this paper introduces a Discriminative Feature Enhancement (DFE) mechanism to further explore the feature discrepancies introduced by various data augmentation strategies across multiple views. By leveraging feature similarity and dissimilarity across multi-scale augmentation techniques, the method reconstructs and models the features, thereby effectively optimizing the salient regions. Furthermore, a prompt generator is developed that integrates Physical Constraints with a Sliding Window (PCSW) mechanism to generate input prompts for unlabeled data, fulfilling SAM-2's requirement for additional prompts. Extensive experiments demonstrate the superiority of the proposed method for semi-supervised medical image segmentation on two multi-label datasets, i.e., ACDC and BHSD. Notably, SSS achieves an average Dice score of 53.15 on BHSD, surpassing the previous state-of-the-art method by +3.65 Dice. Code will be available at https://github.com/AIGeeksGroup/SSS.

Problem

• Leveraging unlabeled data for medical image segmentation
• Enhancing feature support using SAM-2's prior knowledge
• Improving segmentation accuracy with semi-supervised learning

Innovation

• Semi-Supervised SAM-2 leverages unlabeled medical data
• Discriminative Feature Enhancement optimizes multi-view features
• Prompt generator integrates Physical Constraints with Sliding Window