🤖 AI Summary
To address the performance bottleneck of deep learning in skin lesion classification caused by scarce labeled data, this paper proposes a semi-supervised online knowledge distillation framework. The method integrates ensemble learning with online knowledge distillation, enabling multiple convolutional neural networks to collaboratively train using only a small number of labeled samples and abundant unlabeled data via real-time mutual distillation—without requiring additional manual annotations. Its core innovation lies in a self-augmented bidirectional knowledge transfer mechanism, allowing a single student model to approximate ensemble-level inference performance. On the ISIC 2018 and 2019 benchmarks, the single-model accuracy significantly surpasses that of independently trained baselines and approaches the performance of fully supervised ensembles, substantially reducing dependency on labeled data and deployment resource overhead.
📝 Abstract
Deep Learning has emerged as a promising approach for skin lesion analysis. However, existing methods mostly rely on fully supervised learning, requiring extensive labeled data, which is challenging and costly to obtain. To alleviate this annotation burden, this study introduces a novel semi-supervised deep learning approach that integrates ensemble learning with online knowledge distillation for enhanced skin lesion classification. Our methodology involves training an ensemble of convolutional neural network models, using online knowledge distillation to transfer insights from the ensemble to its members. This process aims to enhance the performance of each model within the ensemble, thereby elevating the overall performance of the ensemble itself. Post-training, any individual model within the ensemble can be deployed at test time, as each member is trained to deliver comparable performance to the ensemble. This is particularly beneficial in resource-constrained environments. Experimental results demonstrate that the knowledge-distilled individual model performs better than independently trained models. Our approach demonstrates superior performance on both the emph{International Skin Imaging Collaboration} 2018 and 2019 public benchmark datasets, surpassing current state-of-the-art results. By leveraging ensemble learning and online knowledge distillation, our method reduces the need for extensive labeled data while providing a more resource-efficient solution for skin lesion classification in real-world scenarios.