Explainable Knowledge Distillation for Efficient Medical Image Classification

2025-08-21

AI Summary

To address the dual challenges of efficiency and interpretability in diagnosing COVID-19 and lung cancer from chest X-rays (CXRs) under resource-constrained clinical settings, this work proposes a hybrid-supervised knowledge distillation framework. Methodologically, it employs high-capacity teacher models—VGG19, Visformer-S, and AutoFormer-V2-T—and constructs a hardware-aware lightweight student model based on the OFA-595 supernet. Training integrates joint optimization with both soft and ground-truth labels, while Score-CAM enables spatially interpretable decision visualization. Experiments on the COVID-QU-Ex and LCS25000 datasets demonstrate that the student model reduces parameter count by over 80%, achieves 3.2× inference speedup, and attains classification accuracy only 1.3% lower than the teachers. Moreover, it exhibits strong localization consistency and clinically trustworthy interpretability. These advances significantly enhance the deployability of lightweight models in primary healthcare settings.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This study comprehensively explores knowledge distillation frameworks for COVID-19 and lung cancer classification using chest X-ray (CXR) images. We employ high-capacity teacher models, including VGG19 and lightweight Vision Transformers (Visformer-S and AutoFormer-V2-T), to guide the training of a compact, hardware-aware student model derived from the OFA-595 supernet. Our approach leverages hybrid supervision, combining ground-truth labels with teacher models' soft targets to balance accuracy and computational efficiency. We validate our models on two benchmark datasets: COVID-QU-Ex and LCS25000, covering multiple classes, including COVID-19, healthy, non-COVID pneumonia, lung, and colon cancer. To interpret the spatial focus of the models, we employ Score-CAM-based visualizations, which provide insight into the reasoning process of both teacher and student networks. The results demonstrate that the distilled student model maintains high classification performance with significantly reduced parameters and inference time, making it an optimal choice in resource-constrained clinical environments. Our work underscores the importance of combining model efficiency with explainability for practical, trustworthy medical AI solutions.

Problem

• Develop efficient medical image classification models
• Balance accuracy and computational efficiency in AI
• Provide explainable AI for clinical trustworthiness

Innovation

• Knowledge distillation with teacher-student models
• Hybrid supervision combining labels and soft targets
• Score-CAM visualization for model interpretability