When LLMs Analyze Scars: From Images to Clinically-Meaningful Features

2026-06-16

AI Summary

This work addresses the challenge of data scarcity in classifying keloids and hypertrophic scars—stemming from high annotation costs, privacy constraints, and the rarity of cases—by proposing ScaFE, a novel framework that leverages large language models (LLMs) as knowledge-driven feature engineers. Specifically, ScaFE prompts LLMs to generate executable Python code aligned with clinical standards such as the Vancouver Scar Scale, thereby extracting low-dimensional, interpretable clinical features directly from images. This approach decouples medical knowledge acquisition from statistical learning, enhancing data efficiency and model transparency while preserving patient privacy. Experimental results demonstrate that ScaFE outperforms both end-to-end deep learning models and black-box LLM classifiers under limited data regimes, offering a more efficient, interpretable, and privacy-preserving solution for scar classification.

Abstract

Medical image classification faces a fundamental dilemma: while deep learning models achieve remarkable performance at scale, real-world clinical scenarios often suffer from severe data scarcity due to annotation costs, privacy constraints, and disease rarity. This challenge is particularly pronounced in pathological scar classification, where differentiating keloids from hypertrophic scars requires subtle expert knowledge and labeled images are extremely limited. We propose a novel paradigm that repositions large language models (LLMs) as knowledge-driven feature engineers rather than end-to-end classifiers. We call this framework ScaFE (Scar Feature Engineering). Our key insight is that LLMs encode rich medical knowledge that can be externalized as executable feature extraction code, enabling the transformation of high-dimensional images into low-dimensional, clinically interpretable representations. Specifically, we prompt an LLM with established scar assessment criteria to generate deterministic Python code that extracts features aligned with clinical scoring systems such as the Vancouver Scar Scale. Our approach offers three key advantages: (1) data efficiency, achieving robust performance with limited training samples by decoupling knowledge acquisition from statistical learning; (2) privacy preservation, as raw images are processed locally without exposure to external LLMs; and (3) interpretability, through explicit features grounded in clinical reasoning. Extensive experiments on scar classification demonstrate that our method consistently outperforms end-to-end deep learning baselines or using LLMs as black-box classifiers under limited data conditions, establishing a promising direction for integrating LLMs into data-efficient and clinically transparent medical AI systems.

Problem

• scar classification
• data scarcity
• medical image analysis
• keloid
• hypertrophic scar

Innovation

• Large Language Models
• Feature Engineering
• Medical Image Classification
• Data Efficiency
• Clinical Interpretability