Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets

2024-01-25

Scientific Data

AI Summary

This study systematically audits quality deficiencies in two leading dermatological image datasets—DermaMNIST and Fitzpatrick17k—focusing on label noise, image blurriness, lesion invisibility, and inconsistent Fitzpatrick skin-type annotations. We introduce the first reproducible, multi-dimensional evaluation framework integrating image sharpness analysis, label confidence calibration, lesion segmentation validation, and cross-dataset statistical analysis of skin-tone distributions. Results reveal that 23% of samples exhibit severe label errors or non-visible lesions, while Fitzpatrick skin-type misclassification reaches 18.7%. Beyond quantifying structural limitations undermining clinical generalizability, this work establishes a standardized quality assessment paradigm and provides empirically grounded correction strategies. By identifying and characterizing dataset-level biases and inaccuracies, our approach delivers a foundational data benchmark for developing robust, clinically deployable dermatological AI systems.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Security and Privacy: Data transparency and provenance

Problem

• Data Quality
• Deep Learning
• Skin Image Databases

Innovation

• Deep Learning
• Skin Image Databases
• Data Quality Improvement