Meta-Transfer Derm-Diagnosis: Exploring Few-Shot Learning and Transfer Learning for Skin Disease Classification in Long-Tail Distribution

2024-04-25

IEEE journal of biomedical and health informatics

AI Summary

Addressing the dual challenges of label scarcity and long-tailed class distribution in rare dermatological disease diagnosis, this work presents the first systematic, cross-paradigm evaluation of three few-shot learning approaches—transfer learning, episodic learning, and contrastive self-supervised pretraining—under realistic long-tailed skin disease settings. We propose a novel transfer-learning framework synergizing batch-level mixed-sample augmentation (MixUp, CutMix, and ResizeMix) to enhance generalization from limited labeled examples. Our method achieves state-of-the-art performance on SD-198 and Derm7pt, and strong results on ISIC2018. Extensive experiments span diverse backbones—including MobileNetV2 and ViT—as well as multiple few-shot architectures, ensuring robustness and broad applicability. All code will be publicly released to facilitate reproducibility and further research.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Building accurate models for rare skin diseases remains challenging due to the lack of sufficient labeled data and the inherently long-tailed distribution of available samples. These issues are further complicated by inconsistencies in how datasets are collected and their varying objectives. To address these challenges, we compare three learning strategies: episodic learning, supervised transfer learning, and contrastive self-supervised pretraining, within a few-shot learning framework. We evaluate five training setups on three benchmark datasets: ISIC2018, Derm7pt, and SD-198. Our findings show that traditional transfer learning approaches, particularly those based on MobileNetV2 and Vision Transformer (ViT) architectures, consistently outperform episodic and self-supervised methods as the number of training examples increases. When combined with batch-level data augmentation techniques such as MixUp, CutMix, and ResizeMix, these models achieve state-of-the-art performance on the SD-198 and Derm7pt datasets, and deliver highly competitive results on ISIC2018. All the source codes related to this work will be made publicly available soon at the provided URL.

Problem

• Addressing rare skin disease classification with limited labeled data
• Overcoming long-tailed distribution challenges in dermatological datasets
• Comparing few-shot learning and transfer learning strategies

Innovation

• Combining transfer learning with few-shot learning
• Applying data augmentation techniques like MixUp
• Using MobileNetV2 and Vision Transformer architectures