Towards Interactive Lesion Segmentation in Whole-Body PET/CT with Promptable Models

2025-08-29

AI Summary

Automatic lesion segmentation in whole-body PET/CT suffers from poor robustness due to tracer heterogeneity, physiological uptake variations, and multi-center discrepancies. Method: We propose a clinically oriented promptable segmentation framework featuring: (1) Euclidean distance transform (EDT)-based spatial prompt encoding—replacing conventional Gaussian kernels—to more accurately model foreground/background priors from physician clicks; (2) an online user interaction simulation mechanism with customized point-sampling strategies to enhance generalization to real human–computer collaborative scenarios; and (3) integration of EDT prompting, interaction simulation, and multi-tracer joint training into the nnU-Net architecture. Results: Extensive cross-center validation demonstrates significant reductions in false-positive and false-negative rates, with superior segmentation performance over state-of-the-art baselines. The framework validates both clinical efficacy and practical potential in real-world human–AI collaborative workflows.

Technology Category

• Computer Vision: Segmentation
• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Whole-body PET/CT is a cornerstone of oncological imaging, yet accurate lesion segmentation remains challenging due to tracer heterogeneity, physiological uptake, and multi-center variability. While fully automated methods have advanced substantially, clinical practice benefits from approaches that keep humans in the loop to efficiently refine predicted masks. The autoPET/CT IV challenge addresses this need by introducing interactive segmentation tasks based on simulated user prompts. In this work, we present our submission to Task 1. Building on the winning autoPET III nnU-Net pipeline, we extend the framework with promptable capabilities by encoding user-provided foreground and background clicks as additional input channels. We systematically investigate representations for spatial prompts and demonstrate that Euclidean Distance Transform (EDT) encodings consistently outperform Gaussian kernels. Furthermore, we propose online simulation of user interactions and a custom point sampling strategy to improve robustness under realistic prompting conditions. Our ensemble of EDT-based models, trained with and without external data, achieves the strongest cross-validation performance, reducing both false positives and false negatives compared to baseline models. These results highlight the potential of promptable models to enable efficient, user-guided segmentation workflows in multi-tracer, multi-center PET/CT. Code is publicly available at https://github.com/MIC-DKFZ/autoPET-interactive

Problem

• Enabling interactive lesion segmentation in whole-body PET/CT
• Addressing tracer heterogeneity and multi-center variability challenges
• Developing promptable models with user-guided refinement capabilities

Innovation

• Promptable models with user clicks as input channels
• Euclidean Distance Transform encoding for spatial prompts
• Online simulation and custom point sampling strategy