Pseudo-Text-Conditioned 3D Grounding DINO for Organ Localization in Abdominal CT

2026-06-25

AI Summary

This work addresses the challenge of 3D localization of key abdominal organs in computed tomography (CT) by proposing CT-3GDINO, a lightweight 3D detector. Built upon the Grounding DINO architecture, the method introduces a novel pseudo-text conditioning mechanism that replaces the conventional text encoder with frozen pseudo-text category tokens, thereby establishing an open-vocabulary detection framework without requiring a trainable text encoder. The model integrates a Swin3D visual backbone, bidirectional feature enhancement, pseudo-text-guided query selection, and a cross-modal decoder to enable end-to-end prediction of normalized 3D bounding boxes for the liver, spleen, both kidneys, and intestines. Trained from scratch on 193 RSNA/RATIC CT scans, the multi-scale variant achieves a top-1 class-level mAP of 0.5830 across IoU thresholds from 0.1 to 0.7, with coarse localization (IoU 0.1) reaching 0.9649 AP—significantly outperforming pretrained alternatives—and establishes the first open-source baseline for this task.

Abstract

Reliable organ localization in abdominal CT can provide spatial priors for downstream trauma analysis. We propose CT-3GDINO, a lightweight 3D detector that adapts a Grounding-DINO-style query-based architecture to fixed organ localization using frozen pseudo-text class tokens instead of a real text encoder. The model combines a Swin3D visual backbone, bidirectional feature enhancement, pseudo-text-guided query selection, and a cross-modality decoder to predict normalized 3D boxes for liver, spleen, left kidney, right kidney, and bowel. We train and evaluate on 193 matched RSNA/RATIC CT volumes with segmentation-derived boxes. The best multi-scale model, trained from scratch, achieves 0.5830 overall top-1 class-wise mAP over 3D IoU thresholds from 0.1 to 0.7, outperforming fixed- and trainable-backbone classification-pretrained variants with 0.5570 and 0.4657 mAP. Performance is strong for coarse localization, with 0.9649 AP at IoU 0.1, but remains limited for strict box alignment, with 0.1552 AP at IoU 0.7. These results establish CT-3GDINO as an open-source baseline for pseudo-text-conditioned 3D organ localization and motivate future work on localization-aware pretraining, richer multimodal conditioning, and injury-focused detection.

Problem

• organ localization
• abdominal CT
• 3D grounding
• pseudo-text conditioning
• medical image analysis

Innovation

• pseudo-text-conditioned
• 3D grounding
• organ localization
• query-based detection
• cross-modality decoder