Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification

2026-01-19

AI Summary

This study evaluates the faithfulness and localization reliability of Grad-CAM for interpreting lung cancer classification in chest CT scans. It presents the first systematic comparison between convolutional architectures (ResNet, DenseNet, EfficientNet) and Vision Transformers (ViT) in terms of explanation consistency, localization accuracy, and robustness to perturbations, thereby elucidating how attention mechanisms influence interpretability. The findings reveal that Grad-CAM yields robust explanations in convolutional models but suffers from distorted visualizations in ViT due to its non-local attention, with significant inter-model discrepancies in localization performance—raising concerns about its clinical generalizability. To address these issues, this work proposes a model-aware interpretability evaluation framework, offering a new perspective toward the trustworthy deployment of medical AI systems.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (Grad-CAM), have become indispensable for visualizing the reasoning process of deep neural networks in medical image analysis. Despite their popularity, the faithfulness and reliability of these heatmap-based explanations remain under scrutiny. This study critically investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification. Using the publicly available IQ-OTH/NCCD dataset, we evaluate five representative architectures: ResNet-50, ResNet-101, DenseNet-161, EfficientNet-B0, and ViT-Base-Patch16-224, to explore model-dependent variations in Grad-CAM interpretability. We introduce a quantitative evaluation framework that combines localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability across architectures. Experimental findings reveal that while Grad-CAM effectively highlights salient tumor regions in most convolutional networks, its interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior. Furthermore, cross-model comparisons indicate substantial variability in saliency localization, implying that Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks. This work exposes critical limitations of current saliency-based XAI approaches in medical imaging and emphasizes the need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to"trust"a model's explanation.

Problem

• Grad-CAM
• faithfulness
• localization reliability
• explainable AI
• lung cancer classification

Innovation

• Grad-CAM faithfulness
• localization reliability
• Vision Transformer interpretability
• explainable AI in medical imaging
• quantitative XAI evaluation