Problem
Research questions and friction points this paper is trying to address.
glioma subregion segmentation
text-guided refinement
vision-language model
medical image segmentation
interactive correction
Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model
🤖 AI Summary
This work addresses the poor generalization of glioma subregion segmentation models and their limited responsiveness to precise clinical text-based corrections by proposing a lightweight, interactive segmentation framework built upon the frozen-weight 3D vision-language foundation model VoxTell. For the first time, the pretrained VoxTell is leveraged for text-instructed tumor segmentation editing: textual prompts encoding target, action, and spatial information are projected via a trainable adapter into the multi-scale decoder’s conditional pathway, enabling semantically controllable refinement without end-to-end retraining. On the BraTS-GLI test set, accurate instructions improved the Dice coefficient from 0.774 to 0.796, and cross-dataset experiments demonstrated statistically significant superiority over null or contradictory instructions (p<0.001).
📝 Abstract
Background: Accurate glioma subregion delineation is important for radiotherapy planning and longitudinal monitoring, but manual contour correction is time-consuming. Models such as nnU-Net may generalize imperfectly and lack clinician-directed text correction.
Purpose: We investigated adapting a three-dimensional (3D) vision-language foundation model for text-guided brain tumor segmentation refinement.
Methods: We developed a lightweight VoxTell-based framework. Pretrained VoxTell generated initial masks. Oracle prompts derived from segmentation errors encoded target, action, location, imaging evidence, edit size, and preservation constraints. Frozen Qwen/VoxTell prompt embeddings were injected through trainable projections into its multiscale decoder conditioning; other weights remained frozen. Training, validation, and testing used 901, 100, and 250 BraTS-GLI cases. Cross-dataset transfer was evaluated on 100 meningioma, metastasis, pediatric tumor, and UPENN-GBM cases.
Results: On the internal test set using post-contrast T1-weighted input, correct instructions improved subregion Dice similarity coefficient (DSC; enhancing tumor, edema, and necrotic/non-enhancing core) from $0.774\pm0.158$ to $0.796\pm0.137$. They outperformed blank prompts ($0.762\pm0.155$; Holm-adjusted $p<0.001$, $d_z=0.71$) and contradictory prompts ($0.770\pm0.163$; $p<0.001$, $d_z=0.48$). In cross-dataset testing, correct instructions improved DSC from $0.527\pm0.287$ to $0.550\pm0.278$ and outperformed contradictory instructions ($0.504\pm0.275$; $p<0.001$, $d_z=0.43$).
Conclusion: A 3D vision-language foundation model can perform instruction-guided refinement of glioma subregion segmentations. Sensitivity to correct, blank, and contradictory prompts suggests text-dependent contour editing rather than nonspecific post-processing, supporting further evaluation as a clinician-in-the-loop tool.
Innovation
Methods, ideas, or system contributions that make the work stand out.
vision-language foundation model
text-guided segmentation refinement
glioma subregion segmentation
instruction-conditioned editing
frozen feature injection
🔎 Similar Papers
