🤖 AI Summary
This work addresses the diagnostic challenges in skin lesion classification arising from class imbalance, inter-class similarity, and intra-class variability by proposing a multimodal approach that fuses dermoscopic images with structured clinical metadata. Visual features are extracted using a Swin Transformer and jointly learned with contextual clinical information. To enhance model reliability and interpretability, the framework incorporates temperature scaling calibration and uncertainty estimation. Evaluated on public datasets, the method achieves an accuracy of 92.55% and a macro F1-score of 91.33%, demonstrating significantly improved recognition of minority classes and reduced calibration error. These advances collectively strengthen the trustworthiness and practical utility of automated dermatological diagnosis systems.
📝 Abstract
Skin lesion classification plays an important role in supporting the early diagnosis of skin cancer. However, automated analysis remains challenging due to class imbalance, inter-class similarity, and intra-class variability in dermoscopic images. This paper proposes a multimodal classification framework that combines Swin Transformer-based image features with structured clinical metadata to improve diagnostic performance through integrated visual-context learning. Experiments on a publicly available dataset show that the proposed model achieves a test accuracy of 92.55% and a macro F1-score of 91.33%, with strong performance across minority classes. Temperature scaling is applied as a post-hoc calibration method, resulting in a reduction in expected calibration error and improving prediction reliability, while uncertainty estimation is incorporated to further assess the confidence of model predictions. Qualitative explainability analysis further shows that the model focuses on lesion regions during inference. Therefore, the results demonstrate that multimodal fusion, combined with calibration and interpretability analysis, provides an effective and trustworthy approach for automated skin lesion classification.