Developing Foundation Models for Universal Segmentation from 3D Whole-Body Positron Emission Tomography

2026-03-12

AI Summary

This study addresses the challenges of low anatomical contrast and high annotation costs in 3D whole-body PET imaging, which hinder the application of deep learning–based segmentation. To overcome these limitations, the authors construct the largest multi-center, multi-tracer 3D whole-body PET dataset to date and propose SegAnyPET—the first general-purpose foundation model tailored for this imaging modality. Built upon a 3D architecture with a prompt-driven mechanism, SegAnyPET enables zero-shot cross-task segmentation and facilitates efficient human-in-the-loop interaction. Experimental results demonstrate that SegAnyPET exhibits strong generalization across diverse disease scenarios, significantly enhancing the clinical applicability of molecular imaging.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Deep Neural Architectures and Foundation Models
• Intelligent Robots: Embodied AI

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Positron emission tomography (PET) is a key nuclear medicine imaging modality that visualizes radiotracer distributions to quantify in vivo physiological and metabolic processes, playing an irreplaceable role in disease management. Despite its clinical importance, the development of deep learning models for quantitative PET image analysis remains severely limited, driven by both the inherent segmentation challenge from PET's paucity of anatomical contrast and the high costs of data acquisition and annotation. To bridge this gap, we develop generalist foundational models for universal segmentation from 3D whole-body PET imaging. We first build the largest and most comprehensive PET dataset to date, comprising 11041 3D whole-body PET scans with 59831 segmentation masks for model development. Based on this dataset, we present SegAnyPET, an innovative foundational model with general-purpose applicability to diverse segmentation tasks. Built on a 3D architecture with a prompt engineering strategy for mask generation, SegAnyPET enables universal and scalable organ and lesion segmentation, supports efficient human correction with minimal effort, and enables a clinical human-in-the-loop workflow. Extensive evaluations on multi-center, multi-tracer, multi-disease datasets demonstrate that SegAnyPET achieves strong zero-shot performance across a wide range of segmentation tasks, highlighting its potential to advance the clinical applications of molecular imaging.

Problem

• PET segmentation
• foundation models
• anatomical contrast
• data annotation
• universal segmentation

Innovation

• foundation model
• universal segmentation
• 3D PET imaging
• prompt engineering
• zero-shot learning