SGP-SAM: Self-Gated Prompting for Transferring 3D Segment Anything Models to Lesion Segmentation

2026-04-19

AI Summary

This work addresses the performance degradation of directly transferring 3D SAM models to lesion segmentation, which stems from weak intermediate feature representations and extreme foreground-background imbalance. To mitigate these issues, the authors propose SGP-SAM, a self-gated prompting framework that introduces a Self-Gated Prompt Module (SGPM) to dynamically activate a multi-scale spatial enhancement mechanism only when necessary, thereby effectively fusing multi-level features. Additionally, they design a Zoom Loss that combines Dice loss with a voxel-balanced Focal loss to strengthen supervision signals for small lesions. Transfer experiments based on SAM-Med3D demonstrate that SGP-SAM improves mDice by 7.3% over the fine-tuned baseline on the MSD liver tumor dataset and consistently enhances performance in brain contrast-enhancing tumor segmentation tasks.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Segmentation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large segmentation foundation models such as the Segment Anything Model (SAM) have reshaped promptable segmentation in natural images, and recent efforts have extended these models to medical images and volumetric settings. However, directly transferring a 3D SAM-style model to lesion segmentation remains challenging due to (i) weak spatial representational capacity for small, irregular targets in intermediate features, and (ii) extreme foreground-background imbalance in 3D volumes.We propose SGP-SAM, a self-gated prompting framework for efficient and effective transfer to 3D lesion segmentation. Our key component, the Self-Gated Prompting Module (SGPM), performs conditional multi-scale spatial enhancement: a lightweight multi-channel gating unit predicts whether the current features require additional multi-scale fusion, and only then activates a Multi-Scale Feature Fusion Block to enrich spatial context. To further address small-lesion learning, we design a Zoom Loss that up-weights lesion-focused supervision by combining Dice and a voxel-balanced focal term.Experiments on MSD Liver Tumor and MSD Brain Tumor (enhancing tumor) show consistent gains over strong transfer baselines based on SAM-Med3D. On MSD Liver Tumor, SGP-SAM improves mDice by 7.3% over fine-tuning.

Problem

• 3D lesion segmentation
• spatial representation
• foreground-background imbalance
• small irregular targets
• medical image segmentation

Innovation

• Self-Gated Prompting
• Multi-Scale Feature Fusion
• Zoom Loss
• 3D Lesion Segmentation
• Segment Anything Model