Dynamic Weight Adjustment for Knowledge Distillation: Leveraging Vision Transformer for High-Accuracy Lung Cancer Detection and Real-Time Deployment

2025-10-23

AI Summary

To address the trade-off among accuracy, robustness, and real-time deployability in lung cancer detection, this paper proposes FuzzyDistillViT-MobileNet—a knowledge distillation framework wherein ViT-B32 serves as the teacher and MobileNet as the student. A dynamic fuzzy logic–driven distillation mechanism adaptively assigns region-level weights, prioritizing high-confidence discriminative regions. Multi-scale image preprocessing integrates Gamma correction, histogram equalization, and wavelet fusion, while a genetic algorithm optimizes the student architecture to balance accuracy and efficiency. A novel dynamic waiting mechanism further stabilizes training. Evaluated on LC25000 and IQOTH/NCCD datasets, the framework achieves 99.16% and 99.54% classification accuracy, respectively—demonstrating superior cross-modal diagnostic robustness and generalization capability compared to existing methods.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This paper presents the FuzzyDistillViT-MobileNet model, a novel approach for lung cancer (LC) classification, leveraging dynamic fuzzy logic-driven knowledge distillation (KD) to address uncertainty and complexity in disease diagnosis. Unlike traditional models that rely on static KD with fixed weights, our method dynamically adjusts the distillation weight using fuzzy logic, enabling the student model to focus on high-confidence regions while reducing attention to ambiguous areas. This dynamic adjustment improves the model ability to handle varying uncertainty levels across different regions of LC images. We employ the Vision Transformer (ViT-B32) as the instructor model, which effectively transfers knowledge to the student model, MobileNet, enhancing the student generalization capabilities. The training process is further optimized using a dynamic wait adjustment mechanism that adapts the training procedure for improved convergence and performance. To enhance image quality, we introduce pixel-level image fusion improvement techniques such as Gamma correction and Histogram Equalization. The processed images (Pix1 and Pix2) are fused using a wavelet-based fusion method to improve image resolution and feature preservation. This fusion method uses the wavedec2 function to standardize images to a 224x224 resolution, decompose them into multi-scale frequency components, and recursively average coefficients at each level for better feature representation. To address computational efficiency, Genetic Algorithm (GA) is used to select the most suitable pre-trained student model from a pool of 12 candidates, balancing model performance with computational cost. The model is evaluated on two datasets, including LC25000 histopathological images (99.16% accuracy) and IQOTH/NCCD CT-scan images (99.54% accuracy), demonstrating robustness across different imaging domains.

Problem

• Dynamic fuzzy logic adjusts distillation weights for lung cancer classification uncertainty
• Vision Transformer transfers knowledge to MobileNet for improved generalization capabilities
• Genetic Algorithm selects optimal student model balancing accuracy and computational efficiency

Innovation

• Dynamic fuzzy logic adjusts distillation weights for uncertainty
• Wavelet fusion with Gamma correction enhances image resolution
• Genetic Algorithm selects optimal student model balancing performance cost