IMA++: ISIC Archive Multi-Annotator Dermoscopic Skin Lesion Segmentation Dataset

2025-12-24

AI Summary

The absence of large-scale, publicly available multi-annotator dermoscopic lesion segmentation (SLS) datasets hinders progress in medical image segmentation with multiple expert annotations. To address this, we introduce the largest open multi-annotator SLS dataset to date, comprising 14,967 dermoscopic images and 17,684 high-quality segmentation masks; notably, 2,394 images are independently annotated by 2–5 dermatology experts. For the first time, we systematically collect fine-grained metadata—including annotator expertise levels and tool preferences—alongside structured metadata acquisition, inter-annotator consistency evaluation, and consensus mask generation. The dataset is derived from ISIC Archive images and provides standardized train/val/test splits, consensus labels, and uncertainty quantification. It enables novel research directions including annotator preference modeling, robust segmentation under annotation variability, multi-annotator learning, and human-AI collaborative diagnosis.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web

Abstract

Multi-annotator medical image segmentation is an important research problem, but requires annotated datasets that are expensive to collect. Dermoscopic skin lesion imaging allows human experts and AI systems to observe morphological structures otherwise not discernable from regular clinical photographs. However, currently there are no large-scale publicly available multi-annotator skin lesion segmentation (SLS) datasets with annotator-labels for dermoscopic skin lesion imaging. We introduce ISIC MultiAnnot++, a large public multi-annotator skin lesion segmentation dataset for images from the ISIC Archive. The final dataset contains 17,684 segmentation masks spanning 14,967 dermoscopic images, where 2,394 dermoscopic images have 2-5 segmentations per image, making it the largest publicly available SLS dataset. Further, metadata about the segmentation, including the annotators' skill level and segmentation tool, is included, enabling research on topics such as annotator-specific preference modeling for segmentation and annotator metadata analysis. We provide an analysis on the characteristics of this dataset, curated data partitions, and consensus segmentation masks.

Problem

• Lack of large-scale multi-annotator skin lesion segmentation datasets
• Need for datasets with annotator metadata like skill level
• Enabling research on annotator-specific modeling and consensus segmentation

Innovation

• Largest public multi-annotator skin lesion segmentation dataset
• Includes annotator metadata like skill level and segmentation tool
• Provides consensus masks and curated data partitions for analysis