R$^{2}$Seg: Training-Free OOD Medical Tumor Segmentation via Anatomical Reasoning and Statistical Rejection

2025-11-16

AI Summary

Medical foundation models for image segmentation often yield fragmented false positives on out-of-distribution (OOD) tumors. To address this, we propose a training-free, test-time adaptive two-stage framework: first, an anatomically grounded localization stage driven by a large language model (LLM) identifies target organs and generates multi-scale regions of interest; second, a frozen biomedical foundation model (BiomedParse) extracts features, followed by a parameter-free two-sample statistical test to automatically reject spurious candidates. This work is the first to integrate LLM-guided anatomical priors with nonparametric statistical rejection—eliminating fine-tuning dependencies while enabling test-time adaptation and avoiding catastrophic forgetting. Evaluated on a multi-center, multimodal OOD tumor segmentation benchmark, our method consistently outperforms strong baselines, achieving significant improvements in Dice score, specificity, and sensitivity.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Foundation models for medical image segmentation struggle under out-of-distribution (OOD) shifts, often producing fragmented false positives on OOD tumors. We introduce R$^{2}$Seg, a training-free framework for robust OOD tumor segmentation that operates via a two-stage Reason-and-Reject process. First, the Reason step employs an LLM-guided anatomical reasoning planner to localize organ anchors and generate multi-scale ROIs. Second, the Reject step applies two-sample statistical testing to candidates generated by a frozen foundation model (BiomedParse) within these ROIs. This statistical rejection filter retains only candidates significantly different from normal tissue, effectively suppressing false positives. Our framework requires no parameter updates, making it compatible with zero-update test-time augmentation and avoiding catastrophic forgetting. On multi-center and multi-modal tumor segmentation benchmarks, R$^{2}$Seg substantially improves Dice, specificity, and sensitivity over strong baselines and the original foundation models. Code are available at https://github.com/Eurekashen/R2Seg.

Problem

• Medical image segmentation fails on out-of-distribution tumors
• Foundation models produce fragmented false positive predictions
• Existing methods lack robustness to distribution shifts

Innovation

• Training-free framework via anatomical reasoning
• Statistical rejection filter suppresses false positives
• No parameter updates with zero-update test-time augmentation