DEAP-3DSAM: Decoder Enhanced and Auto Prompt SAM for 3D Medical Image Segmentation

2025-11-24

AI Summary

To address two key limitations of the Segment Anything Model (SAM) in 3D medical image segmentation—spatial feature degradation caused by pseudo-3D processing and heavy reliance on manual prompts—this paper proposes DEAP-3DSAM. Methodologically: (1) a feature-enhanced decoder is designed to jointly integrate native 3D voxel representations with fine-grained spatial structural cues; (2) a dual-attention prompter is introduced to automatically learn adaptive spatial- and channel-wise prompts, eliminating dependence on expert annotations. Extensive experiments on four abdominal tumor 3D datasets demonstrate state-of-the-art performance: DEAP-3DSAM achieves superior or comparable Dice scores and qualitative segmentation accuracy relative to existing prompt-dependent methods. These results validate the model’s core contributions—effective 3D geometric modeling and fully automated prompt generation—thereby advancing SAM’s applicability to volumetric medical imaging.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

The Segment Anything Model (SAM) has recently demonstrated significant potential in medical image segmentation. Although SAM is primarily trained on 2D images, attempts have been made to apply it to 3D medical image segmentation. However, the pseudo 3D processing used to adapt SAM results in spatial feature loss, limiting its performance. Additionally, most SAM-based methods still rely on manual prompts, which are challenging to implement in real-world scenarios and require extensive external expert knowledge. To address these limitations, we introduce the Decoder Enhanced and Auto Prompt SAM (DEAP-3DSAM) to tackle these limitations. Specifically, we propose a Feature Enhanced Decoder that fuses the original image features with rich and detailed spatial information to enhance spatial features. We also design a Dual Attention Prompter to automatically obtain prompt information through Spatial Attention and Channel Attention. We conduct comprehensive experiments on four public abdominal tumor segmentation datasets. The results indicate that our DEAP-3DSAM achieves state-of-the-art performance in 3D image segmentation, outperforming or matching existing manual prompt methods. Furthermore, both quantitative and qualitative ablation studies confirm the effectiveness of our proposed modules.

Problem

• Enhancing spatial features in 3D medical image segmentation
• Automating prompt generation to eliminate manual intervention
• Improving SAM's performance for volumetric medical data analysis

Innovation

• Enhanced decoder fuses image features with spatial information
• Dual attention prompter automates prompts via spatial and channel attention
• Achieves state-of-the-art 3D medical image segmentation performance