🤖 AI Summary
This work addresses the challenge of evaluating whether deep image classification models rely on task-relevant regions, given their opaque decision-making processes. To this end, the authors propose ReMoDEx, a novel framework that integrates local attribution methods—such as GradCAM++ and Integrated Gradients—with global heatmap clustering to enable systematic analysis from sample-level explanations to dataset-level decision patterns. By standardizing heatmaps, performing similarity-based clustering, and assessing spatial correlations, ReMoDEx uncovers shortcut learning behaviors invisible to conventional evaluation metrics. Applied to a COVID-19 chest X-ray classification task, the framework reveals two dominant model strategies: reliance on either the central thoracic region or image borders. Occlusion experiments confirm the latter as a shortcut, despite the model achieving a test accuracy of 86.27% and an AUC of 0.9624.
📝 Abstract
Deep learning image classifiers achieve strong predictive performance yet remain opaque in how decisions are formed. A model may predict correctly while relying on irrelevant cues, shortcut associations, peripheral structures, or device level artifacts instead of task relevant regions. On large scale datasets this opacity is especially problematic, since inspecting heatmaps one sample at a time cannot scale to thousands of predictions. We propose Relevance Based Model Decision Explainability (ReMoDEx), a framework for systematic, dataset scale assessment of model decision behaviour in image classification. ReMoDEx defines a stepwise pipeline: model inference, target class selection, relevance map generation, heatmap standardisation, similarity based grouping of patterns, cluster level interpretation, and spatial relevance assessment. Local methods GradCAM++, Integrated Gradients, Occlusion Sensitivity, and Layerwise Relevance Propagation are each combined independently with a single global module that summarises an entire set of relevance maps into a few decision strategy clusters, replacing sample by sample inspection with an automatic, scalable summary. To demonstrate ReMoDEx, we applied it to a VGG16 based classifier distinguishing COVID-19, Normal, Lung Opacity, and Viral Pneumonia. The classifier showed stable performance (86.27% test accuracy, 0.9624 test AUC). However, each explainer combined with the global module consistently produced two recurring strategies: central thoracic region decisions and border/corner sensitive decisions, indicating possible shortcut learning that conventional metrics could not reveal. Masked image validation confirmed that model confidence and predicted class changed when central or peripheral regions were occluded. ReMoDEx thus provides a scalable relevance based decision assessment framework and an essential complement to accuracy based evaluation.