Skin-R1: Toward Trustworthy Clinical Reasoning for Dermatological Diagnosis

2025-11-18

AI Summary

Current dermatological vision-language model (VLM) diagnostic systems face three key bottlenecks: heterogeneous data labeling, lack of clinically grounded reasoning justification, and poor generalization. To address these, we propose a textbook-guided hierarchical reinforcement learning framework. Our method innovatively constructs a high-fidelity reasoning trajectory generator that integrates disease taxonomic hierarchies with differential diagnosis logic, and introduces a knowledge-injected supervised fine-tuning (SFT) stage jointly optimized with hierarchical RL—enabling trustworthy reasoning transfer from densely annotated to sparse-data regimes. The approach end-to-end models clinical diagnostic reasoning and achieves significant accuracy gains across multiple public benchmarks. Ablation studies confirm SFT’s critical role in establishing robust foundational reasoning. This work is the first to synergistically incorporate textbook-style deep clinical reasoning and structured reinforcement learning into dermatological VLMs, markedly improving model interpretability, robustness, and clinical utility.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

The emergence of vision-language models (VLMs) has opened new possibilities for clinical reasoning and has shown promising performance in dermatological diagnosis. However, their trustworthiness and clinical utility are often limited by three major factors: (1) Data heterogeneity, where diverse datasets lack consistent diagnostic labels and clinical concept annotations; (2) Absence of grounded diagnostic rationales, leading to a scarcity of reliable reasoning supervision; and (3) Limited scalability and generalization, as models trained on small, densely annotated datasets struggle to transfer nuanced reasoning to large, sparsely-annotated ones. To address these limitations, we propose SkinR1, a novel dermatological VLM that combines deep, textbook-based reasoning with the broad generalization capabilities of reinforcement learning (RL). SkinR1 systematically resolves the key challenges through a unified, end-to-end framework. First, we design a textbook-based reasoning generator that synthesizes high-fidelity, hierarchy-aware, and differential-diagnosis (DDx)-informed trajectories, providing reliable expert-level supervision. Second, we leverage the constructed trajectories for supervised fine-tuning (SFT) empowering the model with grounded reasoning ability. Third, we develop a novel RL paradigm that, by incorporating the hierarchical structure of diseases, effectively transfers these grounded reasoning patterns to large-scale, sparse data. Extensive experiments on multiple dermatology datasets demonstrate that SkinR1 achieves superior diagnostic accuracy. The ablation study demonstrates the importance of the reasoning foundation instilled by SFT.

Problem

• Addressing data heterogeneity with inconsistent diagnostic labels and annotations
• Solving absence of grounded diagnostic rationales for reliable reasoning supervision
• Overcoming limited scalability and generalization in dermatological vision-language models

Innovation

• Textbook-based reasoning generator synthesizes expert diagnostic trajectories
• Supervised fine-tuning with trajectories enables grounded reasoning ability
• Reinforcement learning transfers reasoning to large sparse datasets