A Comprehensive Survey of Medical Image Segmentation: Challenges, Benchmarks, and Beyond

2026-06-14

AI Summary

Medical image segmentation faces significant challenges, including methodological fragmentation, inconsistent evaluation protocols, and difficulties in cross-model comparison. This work presents the first unified analytical framework encompassing the three dominant architectural paradigms: U-Net, Vision Transformers, and the Segment Anything Model (SAM). Through a systematic review of their technical evolution and comprehensive benchmarking on public datasets using multidimensional evaluation metrics, the study offers a thorough comparison of these approaches in terms of both accuracy and computational efficiency. To foster standardization and clinical translation, the authors establish a curated segmentation methodology atlas and release an open-source repository of implementations and evaluation tools, thereby advancing reproducibility and coherence in medical image segmentation research.

Abstract

Medical image segmentation plays a critical role in clinical diagnostics, treatment planning, disease monitoring, and neurological disorder identification. This article presents a comprehensive review of its systematic development, covering widely used public datasets, representative methods built on the U-Net, Transformer, and SAM architectures, and key evaluation metrics with their differences, followed by an analysis of major challenges from multiple perspectives. Unlike surveys that focus on a single model family or a specific clinical application, this review organizes U-Net-, Transformer-, and SAM-based methods within a unified analytical framework, with a particular focus on their effectiveness in improving segmentation accuracy and efficiency. This work aims to guide future research and support clinical translation of medical image segmentation, with all related resources publicly available in our GitHub repository: https://github.com/andrew-pengyu/Awsome_MedSeg/tree/main.

Problem

• medical image segmentation
• clinical diagnostics
• segmentation accuracy
• evaluation metrics
• clinical translation

Innovation

• medical image segmentation
• unified analytical framework
• U-Net
• Transformer
• SAM