vesselFM: A Foundation Model for Universal 3D Blood Vessel Segmentation

2024-11-26

arXiv.org

AI Summary

3D vascular segmentation in multimodal medical imaging suffers from poor generalizability and high annotation costs. To address these challenges, we propose the first foundation model specifically designed for general-purpose 3D vascular segmentation. Our method introduces a novel tri-source collaborative training paradigm—uniquely integrating real expert annotations, domain-randomized synthetic data, and flow-matching–generated data—to overcome modality- and annotation-specific limitations. Built upon a foundation model architecture, the framework incorporates domain randomization, continuous flow-matching generative modeling, and joint optimization over heterogeneous data sources. Evaluated across four clinical modalities—CTA, MRA, DSA, and Micro-CT—the model achieves state-of-the-art performance under zero-shot, one-shot, and few-shot settings, significantly outperforming existing medical segmentation foundation models. Notably, it is the first to demonstrate robust cross-modal and cross-protocol zero-shot generalization without fine-tuning.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: Other Foundations of Cognitive Modeling & Systems

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Segmenting 3D blood vessels is a critical yet challenging task in medical image analysis. This is due to significant imaging modality-specific variations in artifacts, vascular patterns and scales, signal-to-noise ratios, and background tissues. These variations, along with domain gaps arising from varying imaging protocols, limit the generalization of existing supervised learning-based methods, requiring tedious voxel-level annotations for each dataset separately. While foundation models promise to alleviate this limitation, they typically fail to generalize to the task of blood vessel segmentation, posing a unique, complex problem. In this work, we present vesselFM, a foundation model designed specifically for the broad task of 3D blood vessel segmentation. Unlike previous models, vesselFM can effortlessly generalize to unseen domains. To achieve zero-shot generalization, we train vesselFM on three heterogeneous data sources: a large, curated annotated dataset, data generated by a domain randomization scheme, and data sampled from a flow matching-based generative model. Extensive evaluations show that vesselFM outperforms state-of-the-art medical image segmentation foundation models across four (pre-)clinically relevant imaging modalities in zero-, one-, and few-shot scenarios, therefore providing a universal solution for 3D blood vessel segmentation.

Problem

• Generalizing 3D blood vessel segmentation across imaging modalities
• Overcoming domain gaps in medical image analysis
• Reducing reliance on voxel-level annotations for segmentation

Innovation

• vesselFM: foundation model for 3D vessel segmentation
• Trained on heterogeneous data for zero-shot generalization
• Outperforms state-of-the-art across multiple imaging modalities