🤖 AI Summary
Deep learning models in medical image diagnosis suffer from limited interpretability, undermining clinical trust. To address this, we conduct the first systematic cross-model, cross-method, and multi-dataset explainable AI (XAI) validation for glaucoma fundus image classification. We evaluate five class activation mapping (CAM) techniques—including Grad-CAM—across four architectures (VGG-11, ResNet-18, DeiT-Tiny, Swin-Tiny) on five publicly available datasets. Attention heatmaps are quantitatively aligned with clinically relevant anatomical structures (optic cup, optic disc, retinal vessels). Paired t-tests and Pearson/Spearman correlation analyses demonstrate that all models significantly attend to these regions (p < 0.001), and the anatomical coverage ratio of attention maps strongly correlates with diagnostic accuracy (r > 0.8, p < 0.001). This work provides the first statistical evidence—spanning multiple models, explanation methods, and datasets—supporting the clinical credibility of XAI in ophthalmic diagnosis.
📝 Abstract
While deep learning has exhibited remarkable predictive capabilities in various medical image tasks, its inherent black-box nature has hindered its widespread implementation in real-world healthcare settings. Our objective is to unveil the decision-making processes of deep learning models in the context of glaucoma classification by employing several Class Activation Map (CAM) techniques to generate model focus regions and comparing them with clinical domain knowledge of the anatomical area (optic cup, optic disk, and blood vessels). Four deep neural networks, including VGG-11, ResNet-18, DeiT-Tiny, and Swin Transformer-Tiny, were developed using binary diagnostic labels of glaucoma and five CAM methods (Grad-CAM, XGrad-CAM, Score-CAM, Eigen-CAM, and Layer-CAM) were employed to highlight the model focus area. We applied the paired-sample t-test to compare the percentage of anatomies in the model focus area to the proportion of anatomies in the entire image. After that, Pearson's and Spearman's correlation tests were implemented to examine the relationship between model predictive ability and the percentage of anatomical structures in the model focus area. On five public glaucoma datasets, all deep learning models consistently displayed statistically significantly higher percentages of anatomical structures in the focus area than the proportions of anatomical structures in the entire image. Also, we validated the positive relationship between the percentage of anatomical structures in the focus area and model predictive performance. Our study provides evidence of the convergence of decision logic between deep neural networks and human clinicians through rigorous statistical tests. We anticipate that it can help alleviate clinicians' concerns regarding the trustworthiness of deep learning in healthcare. For reproducibility, the code and dataset have been released at GitHub.