GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosis

2026-06-29

AI Summary

This study addresses three critical clinical limitations of existing prototypical medical image classifiers: neglecting anatomical relationships among lesions, amplifying physician annotation errors, and requiring full retraining for new lesion types. To overcome these issues, the authors propose a unified architecture that models lesion co-occurrence via a graph attention task head, incorporates a concept-mismatch safety check to filter erroneous feedback, and leverages open-vocabulary prototype anchoring to enable single-shot integration of novel lesions without retraining. Notably, this work introduces safety verification into prototypical classifiers for the first time, substantially improving diagnostic consistency and interactivity. Experiments demonstrate a 13.8% macro F1 improvement on TBX11K, 85% detection of mislabeled annotations by the safety check, single-shot Effusion localization matching fully supervised accuracy, and 2.6× better lesion localization than end-to-end baselines with only ~1 ms added latency.

Abstract

Prototype-based medical image classifiers present three clinical limitations: they treat findings as independent, silently amplify unsafe physician feedback, and require full retraining whenever a new finding is needed. We present GRAPE (Graph-Augmented Prototype Explanations), a unified architecture that addresses all three challenges. First, a Graph Attention Task Head models anatomical concept co-occurrence, boosting macro-F1 by +13.8,pp over the prototype baseline on TBX11K. Second, a Concept-Mismatch Safety Check - the first such mechanism in prototype-based medical classifiers - warns when the model's dominant finding inside a doctor-drawn region conflicts with the claimed label, catching 85% of erroneous annotations versus 51% for MC-Dropout with no extra inference cost. Third, Open-Vocabulary Prototype Anchoring aligns visual prototypes to clinical text, allowing a new finding to be added from a single labeled image without modifying any other component. On NIH ChestX-ray14, one Effusion example recovers full-supervision localization accuracy; on TBX11K, prototype maps achieve 2.6x better lesion localization than end-to-end baselines. All three capabilities add only +1~ms latency at interactive batch size. The project page is https://github.com/KurbanIntelligenceLab/GRAPE.

Problem

• prototype-based medical image classification
• clinical limitations
• anatomical concept co-occurrence
• unsafe physician feedback
• model retraining

Innovation

• Graph Attention
• Prototype-based Explanation
• Concept-Mismatch Safety Check
• Open-Vocabulary Learning
• Medical Image Diagnosis