Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis

2025-07-23

AI Summary

Evaluating fairness of AI skin cancer classifiers—particularly for melanoma—across gender, age, and race remains challenging due to severe underrepresentation of minority groups in real-world clinical datasets. Method: To address data scarcity and bias, we introduce a novel synthetic-data paradigm leveraging generative AI, specifically the controllable diffusion model LightningDiT, to generate high-fidelity skin lesion images annotated with multi-attribute labels (skin tone, age, gender). Contribution/Results: Our cross-distribution fairness evaluation demonstrates that synthetic data enables robust, quantitative fairness assessment of dermatological AI models. Critically, even when training data originates from distributions distinct from the synthetic data, our approach reliably uncovers significant performance disparities across demographic subgroups. This work establishes a scalable, privacy-preserving, and attribute-controllable benchmark framework for fairness evaluation in medical AI.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Security and Privacy: Data transparency and provenance

Abstract

Recent advancements in Deep Learning and its application on the edge hold great potential for the revolution of routine screenings for skin cancers like Melanoma. Along with the anticipated benefits of this technology, potential dangers arise from unforseen and inherent biases. Thus, assessing and improving the fairness of such systems is of utmost importance. A key challenge in fairness assessment is to ensure that the evaluation dataset is sufficiently representative of different Personal Identifiable Information (PII) (sex, age, and race) and other minority groups. Against the backdrop of this challenge, this study leverages the state-of-the-art Generative AI (GenAI) LightningDiT model to assess the fairness of publicly available melanoma classifiers. The results suggest that fairness assessment using highly realistic synthetic data is a promising direction. Yet, our findings indicate that verifying fairness becomes difficult when the melanoma-detection model used for evaluation is trained on data that differ from the dataset underpinning the synthetic images. Nonetheless, we propose that our approach offers a valuable new avenue for employing synthetic data to gauge and enhance fairness in medical-imaging GenAI systems.

Problem

• Assessing fairness of AI skin lesion classifiers
• Ensuring representative datasets for diverse groups
• Using GenAI synthetic data for fairness evaluation

Innovation

• Uses GenAI LightningDiT for synthetic image generation
• Assesses fairness of melanoma classifiers with synthetic data
• Proposes synthetic data for medical-imaging fairness enhancement