How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?

2025-01-20

International Conference on Learning Representations

AI Summary

Limited large-scale annotated data hinders effective pretraining for 3D medical image segmentation. Method: This paper introduces AbdomenAtlas 1.1—a high-quality abdominal CT dataset comprising 9,262 cases, covering 25 anatomical structures and seven tumor classes with pseudo-labels—and proposes an efficient supervised 3D pretraining paradigm based on a 3D U-Net variant. It integrates voxel-level supervision, cross-task transfer, pseudo-label augmentation, and standardized collaborative annotation. Contribution/Results: The first systematic validation demonstrates that supervised 3D pretraining significantly improves downstream segmentation performance. Remarkably, only 21 finely annotated cases suffice to match the performance of unsupervised pretraining on 5,050 cases. Under few-shot settings (21 cases / 672 masks / 40 GPU-hours), our method surpasses existing pretrained models across all metrics and achieves comparable accuracy to large-scale unsupervised approaches (1,152 GPU-hours), substantially enhancing data efficiency and clinical deployability.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Large Multimodal Models (LMMs)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

The pre-training and fine-tuning paradigm has become prominent in transfer learning. For example, if the model is pre-trained on ImageNet and then fine-tuned to PASCAL, it can significantly outperform that trained on PASCAL from scratch. While ImageNet pre-training has shown enormous success, it is formed in 2D, and the learned features are for classification tasks; when transferring to more diverse tasks, like 3D image segmentation, its performance is inevitably compromised due to the deviation from the original ImageNet context. A significant challenge lies in the lack of large, annotated 3D datasets rivaling the scale of ImageNet for model pre-training. To overcome this challenge, we make two contributions. Firstly, we construct AbdomenAtlas 1.1 that comprises 9,262 three-dimensional computed tomography (CT) volumes with high-quality, per-voxel annotations of 25 anatomical structures and pseudo annotations of seven tumor types. Secondly, we develop a suite of models that are pre-trained on our AbdomenAtlas 1.1 for transfer learning. Our preliminary analyses indicate that the model trained only with 21 CT volumes, 672 masks, and 40 GPU hours has a transfer learning ability similar to the model trained with 5,050 (unlabeled) CT volumes and 1,152 GPU hours. More importantly, the transfer learning ability of supervised models can further scale up with larger annotated datasets, achieving significantly better performance than preexisting pre-trained models, irrespective of their pre-training methodologies or data sources. We hope this study can facilitate collective efforts in constructing larger 3D medical datasets and more releases of supervised pre-trained models.

Problem

• 3D Model Evaluation
• Medical Image Analysis
• Pretraining Challenges

Innovation

• 3D Pre-training
• Large-scale Medical Dataset
• Enhanced Model Performance