Automated Disentangling Analysis of Skin Colour for Lesion Images

2026-02-22

AI Summary

This work addresses the degradation in generalization performance of skin lesion classification models caused by variations in skin tone—confounded by factors such as illumination, camera settings, and individual differences—between training and deployment. To mitigate this, the authors propose a compression-based disentangled representation framework that learns a structured and controllable latent space for skin tone from unlabeled dermoscopic images. The method innovatively employs a stochastic monotonic desaturation mapping to prevent leakage of dark-pigment features and integrates a geometric alignment post-processing step to preserve local color patterns, enabling physically meaningful counterfactual skin tone editing and cross-image transfer. Experiments demonstrate that the approach not only enables high-fidelity skin tone manipulation but also significantly improves lesion classification accuracy, particularly when used for data augmentation and skin tone normalization.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Machine-learning models working on skin images often have degraded performance when the skin colour captured in images (SCCI) differs between training and deployment. Such differences arise from entangled environmental factors (e.g., illumination, camera settings), and intrinsic factors (e.g., skin tone) that cannot be accurately described by a single "skin tone" scalar. To mitigate such colour mismatch, we propose a skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images. To prevent information leakage that hinders proper learning of dark colour features, we introduce a randomized, mostly monotonic decolourization mapping. To suppress unintended colour shifts of localized patterns (e.g., ink marks, scars) during colour manipulation, we further propose a geometry-aligned post-processing step. Together, these components enable faithful counterfactual editing and answering an essential question: "What would this skin condition look like under a different SCCI?", as well as direct colour transfer between images and controlled traversal along physically meaningful directions (e.g., blood perfusion, camera white balance), enabling educational visualization of skin conditions under varying SCCI. We demonstrate that dataset-level augmentation and colour normalization based on our framework achieve competitive lesion classification performance.

Problem

• skin colour
• lesion images
• colour mismatch
• machine learning
• disentanglement

Innovation

• disentanglement-by-compression
• skin colour disentangling
• counterfactual editing
• geometry-aligned post-processing
• unlabelled dermatology images