🤖 AI Summary
This study investigates the feasibility of leveraging a frozen DINOv2 ViT-H/16 foundation vision model for multi-label detection of tumors and cysts in 3D kidney CT scans without domain-specific pretraining. Using the KiTS23 dataset, the authors systematically evaluate three patch token aggregation strategies: CLS linear probing, gated attention-based multiple instance learning (MIL), and ProtoViT prototype heads. Results show that attention MIL achieves AUROCs of 0.74 and 0.80 for tumor and cyst detection, respectively, and demonstrates strong spatial interpretability—its attention weights concentrate 7.5–9.8× more on true lesion regions. In contrast, the prototype head fails entirely on cyst detection. The work highlights a critical trade-off between performance and interpretability when applying large-scale vision foundation models to medical multi-label classification tasks.
📝 Abstract
Foundation vision models trained on natural images transfer to medical tasks without domain pre-training, but volumetric classification requires aggregating tens of thousands of patch tokens per study, and the aggregator constrains how the resulting model can be interpreted. We compare three aggregators on identical frozen DINOv3 ViT-H/16+ features for renal tumour/cyst detection on KiTS23 (966 kidneys; n=97 test): a CLS-token linear probe, gated attention multiple instance learning (MIL) over 55,296 patch tokens, and a prototype head following ProtoViT. Attention MIL achieves the highest AUROC for tumour (0.74, 95% CI 0.64-0.83) and cyst (0.80, 0.70-0.88), with attention enriched 7.5-9.8x over chance within annotated lesions. The prototype head does not transfer to cyst detection (AUROC 0.51), exposing an interpretability-performance trade-off at this token scale.