RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning

2026-06-23

AI Summary

This work addresses the challenge of false positives in PET/CT lesion segmentation caused by physiological uptake and imaging artifacts. The authors propose a novel paradigm that integrates voxel-level segmentation with lesion-level reasoning via a large language model (LLM). Initially, permissive candidate regions are generated and converted into structured textual descriptions, which are then evaluated by the LLM in conjunction with multimodal clinical context—such as radiology reports—to determine lesion validity. The approach innovatively leverages the LLM for anatomical semantic reasoning and employs Group Relative Policy Optimization, a reinforcement learning strategy, to enhance the model’s consistency and accuracy. Evaluated on the AutoPET and OSU test sets, the method significantly outperforms purely image-based approaches, demonstrating the greatest improvements in false positive suppression and clinical consistency when radiology reports are available.

Abstract

Accurate lesion segmentation in PET/CT is critical for oncology, yet remains challenging because physiologic tracer uptake and artifacts can mimic malignant signal. We present RADIANT-PET, a reasoning-augmented framework that couples a high-sensitivity voxel-level segmentation model with lesion-level large language model (LLM) adjudication. Candidate uptake regions are generated with a deliberately permissive segmentation stage, then converted into structured textual descriptions that summarize uptake intensity, morphology, and regional and global anatomical context. An LLM classifies each candidate as true lesion vs. false positive, optionally leveraging the radiology report as additional clinical context. To strengthen lesion-level reasoning, we further optimize a local LLM via reinforcement learning using Group Relative Policy Optimization, rewarding correct lesion classification and anatomically concordant site assignment. Across AutoPET and an OSU test cohort, RADIANT-PET consistently outperforms strong image-only baselines, with the largest improvements observed when radiology reports are provided. Overall, these results demonstrate that LLM-based lesion-level reasoning adds a novel reasoning layer beyond conventional segmentation, suppressing physiologic false positives and aligning voxel-level predictions with clinical interpretation. The project repository is available at: https://github.com/jwang-580/RADIANT-PET.

Problem

• PET/CT
• lesion segmentation
• false positives
• physiologic uptake
• oncology

Innovation

• reasoning-augmented segmentation
• large language models
• reinforcement learning
• PET/CT lesion segmentation
• clinical context integration