SegVol: Universal and Interactive Volumetric Medical Image Segmentation

2023-11-22

Neural Information Processing Systems

AI Summary

Existing 3D medical image segmentation models lack generalizability across anatomical categories under simple user interaction. This work introduces the first 3D medical foundation segmentation model supporting both semantic and spatial prompting, covering over 200 anatomical structures in volumetric modalities such as CT. Methodologically, we propose a zoom-out-zoom-in mechanism to enhance 3D inference efficiency; integrate a 3D vision foundation architecture with self-supervised pretraining, prompt-driven segmentation, multi-scale voxel attention, and hierarchical spatial focusing; and train the unified model on 90K unlabeled and 6K labeled CT volumes. Evaluated on 22 anatomical segmentation tasks, our model achieves state-of-the-art performance on 19, with up to a 37.24% relative improvement in Dice score. The model and code are publicly released.

Abstract

Precise image segmentation provides clinical study with instructive information. Despite the remarkable progress achieved in medical image segmentation, there is still an absence of a 3D foundation segmentation model that can segment a wide range of anatomical categories with easy user interaction. In this paper, we propose a 3D foundation segmentation model, named SegVol, supporting universal and interactive volumetric medical image segmentation. By scaling up training data to 90K unlabeled Computed Tomography (CT) volumes and 6K labeled CT volumes, this foundation model supports the segmentation of over 200 anatomical categories using semantic and spatial prompts. To facilitate efficient and precise inference on volumetric images, we design a zoom-out-zoom-in mechanism. Extensive experiments on 22 anatomical segmentation tasks verify that SegVol outperforms the competitors in 19 tasks, with improvements up to 37.24% compared to the runner-up methods. We demonstrate the effectiveness and importance of specific designs by ablation study. We expect this foundation model can promote the development of volumetric medical image analysis. The model and code are publicly available at: https://github.com/BAAI-DCAI/SegVol.

Problem

• 3D foundation segmentation model
• universal medical image segmentation
• interactive user prompts

Innovation

• 3D foundation segmentation model
• scales up training data
• zoom-out-zoom-in mechanism