Atlas-Assisted Segment Anything Model for Fetal Brain MRI (FeTal-SAM)

2026-01-22

AI Summary

This work addresses the challenges in fetal brain MRI segmentation, where models often require repeated retraining and struggle to disentangle the contributions of image contrast from spatial priors. To overcome these limitations, the authors propose an atlas-guided prompting mechanism that, for the first time, leverages spatially aligned label templates—generated via multi-atlas registration—as dense prompts alongside bounding box prompts to drive the SAM decoder. This enables on-demand binary segmentation of user-specified structures without any model retraining, significantly enhancing interpretability. The resulting 2D segmentations are fused into a complete 3D volume. Evaluated on the dHCP and an internal dataset, the method achieves Dice scores for high-contrast structures such as the cortical plate and cerebellum that rival those of state-of-the-art task-specific models, demonstrating its potential as a versatile tool for fetal brain segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

This paper presents FeTal-SAM, a novel adaptation of the Segment Anything Model (SAM) tailored for fetal brain MRI segmentation. Traditional deep learning methods often require large annotated datasets for a fixed set of labels, making them inflexible when clinical or research needs change. By integrating atlas-based prompts and foundation-model principles, FeTal-SAM addresses two key limitations in fetal brain MRI segmentation: (1) the need to retrain models for varying label definitions, and (2) the lack of insight into whether segmentations are driven by genuine image contrast or by learned spatial priors. We leverage multi-atlas registration to generate spatially aligned label templates that serve as dense prompts, alongside a bounding-box prompt, for SAM's segmentation decoder. This strategy enables binary segmentation on a per-structure basis, which is subsequently fused to reconstruct the full 3D segmentation volumes. Evaluations on two datasets, the dHCP dataset and an in-house dataset demonstrate FeTal-SAM's robust performance across gestational ages. Notably, it achieves Dice scores comparable to state-of-the-art baselines which were trained for each dataset and label definition for well-contrasted structures like cortical plate and cerebellum, while maintaining the flexibility to segment any user-specified anatomy. Although slightly lower accuracy is observed for subtle, low-contrast structures (e.g., hippocampus, amygdala), our results highlight FeTal-SAM's potential to serve as a general-purpose segmentation model without exhaustive retraining. This method thus constitutes a promising step toward clinically adaptable fetal brain MRI analysis tools.

Problem

• fetal brain MRI
• segmentation flexibility
• label redefinition
• spatial priors
• image contrast

Innovation

• Atlas-based prompting
• Segment Anything Model
• Fetal brain MRI segmentation
• Multi-atlas registration
• Foundation model