A generalizable 3D framework and model for self-supervised learning in medical imaging

2025-01-20

AI Summary

Existing self-supervised methods for 3D medical imaging suffer from simplistic preprocessing, modality- or organ-specific designs, and poor generalizability. To address this, we propose 3DINO—the first general-purpose self-supervised pretraining framework for 3D medical imaging—and the 3DINO-ViT model, trained on a large-scale, heterogeneous dataset of 100,000 cross-modal (CT/MRI) and multi-organ 3D scans. We innovatively adapt the DINO paradigm to 3D medical data by designing a dedicated 3D Vision Transformer architecture, introducing unified multimodal and multi-organ preprocessing and augmentation strategies, and employing contrastive feature consistency optimization. Extensive evaluations demonstrate state-of-the-art performance across segmentation and classification tasks, with particularly pronounced gains in low-data and out-of-distribution settings. To foster community advancement, we will open-source the model, thereby accelerating research on foundational 3D medical models and their downstream adaptation.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Unsupervised & Self-Supervised Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Current self-supervised learning methods for 3D medical imaging rely on simple pretext formulations and organ- or modality-specific datasets, limiting their generalizability and scalability. We present 3DINO, a cutting-edge SSL method adapted to 3D datasets, and use it to pretrain 3DINO-ViT: a general-purpose medical imaging model, on an exceptionally large, multimodal, and multi-organ dataset of ~100,000 3D medical imaging scans from over 10 organs. We validate 3DINO-ViT using extensive experiments on numerous medical imaging segmentation and classification tasks. Our results demonstrate that 3DINO-ViT generalizes across modalities and organs, including out-of-distribution tasks and datasets, outperforming state-of-the-art methods on the majority of evaluation metrics and labeled dataset sizes. Our 3DINO framework and 3DINO-ViT will be made available to enable research on 3D foundation models or further finetuning for a wide range of medical imaging applications.

Problem

• 3D medical image
• self-supervised learning
• multi-scenario application

Innovation

• 3DINO
• self-supervised learning
• 3D medical images