Fully Automated SAM for Single-source Domain Generalization in Medical Image Segmentation

2025-07-23

AI Summary

Existing SAM-based single-source domain generalization methods for medical image segmentation rely on expert-provided prompts and are highly sensitive to suboptimal prompts (e.g., poorly sized bounding boxes), hindering fully automatic deployment. Method: We propose a prompt-free domain generalization framework built upon SAM, enabling end-to-end unsupervised domain adaptation. It features (1) a collaborative Auto-prompted Generation Model and Shallow Feature Uncertainty Modeling module to robustly generate target-domain bounding box prompts without human input; and (2) an Image-Prompt Embedding Fusion module that adaptively integrates multi-scale image features with prompt embeddings to mitigate prompt-induced noise. Results: Evaluated on public prostate MRI and retinal vessel datasets, our method significantly outperforms state-of-the-art approaches, demonstrating superior robustness to domain shift and low-quality prompts—advancing the clinical feasibility of fully automated medical image segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Although SAM-based single-source domain generalization models for medical image segmentation can mitigate the impact of domain shift on the model in cross-domain scenarios, these models still face two major challenges. First, the segmentation of SAM is highly dependent on domain-specific expert-annotated prompts, which prevents SAM from achieving fully automated medical image segmentation and therefore limits its application in clinical settings. Second, providing poor prompts (such as bounding boxes that are too small or too large) to the SAM prompt encoder can mislead SAM into generating incorrect mask results. Therefore, we propose the FA-SAM, a single-source domain generalization framework for medical image segmentation that achieves fully automated SAM. FA-SAM introduces two key innovations: an Auto-prompted Generation Model (AGM) branch equipped with a Shallow Feature Uncertainty Modeling (SUFM) module, and an Image-Prompt Embedding Fusion (IPEF) module integrated into the SAM mask decoder. Specifically, AGM models the uncertainty distribution of shallow features through the SUFM module to generate bounding box prompts for the target domain, enabling fully automated segmentation with SAM. The IPEF module integrates multiscale information from SAM image embeddings and prompt embeddings to capture global and local details of the target object, enabling SAM to mitigate the impact of poor prompts. Extensive experiments on publicly available prostate and fundus vessel datasets validate the effectiveness of FA-SAM and highlight its potential to address the above challenges.

Problem

• Eliminates need for expert prompts in SAM-based medical segmentation
• Reduces errors from poor prompts in SAM segmentation
• Enhances cross-domain generalization in medical image analysis

Innovation

• Auto-prompted Generation Model with SUFM module
• Image-Prompt Embedding Fusion in SAM decoder
• Uncertainty modeling for automated bounding box prompts