PEFT-MedSAM: Efficient Fine-Tuning of Medical Foundation Models for Explainable Skin Lesion Segmentation

2026-06-17

AI Summary

This study addresses the limited performance of existing deep learning methods in skin lesion segmentation, which hinders early melanoma detection. To overcome this, the authors propose PEFT-MedSAM, the first approach to integrate parameter-efficient fine-tuning (PEFT) into the Medical Segment Anything Model (MedSAM). By fine-tuning only the lightweight mask decoder while freezing both the image and prompt encoders, PEFT-MedSAM achieves an optimal balance between computational efficiency and model interpretability. The method further enhances clinical trustworthiness through Grad-CAM visualizations and pointing game evaluations. On the ISIC 2018 dataset, PEFT-MedSAM attains a Dice score of 0.9411 and an IoU of 0.8918, with external validation on PH2 yielding a Dice of 0.9467—significantly outperforming both U-Net and zero-shot MedSAM (p < 0.0001).

Abstract

Automated segmentation of skin lesions using deep learning models for dermoscopic images can be very helpful in finding melanomas earlier than they would normally be detected. However, most deep learning methods available do not perform well. The aim of this paper is to present a parameter-efficient fine-tuning method called PEFT-MedSAM for adapting the Medical Segment Anything Model (MedSAM) to automatically segment dermoscopic skin lesions. The PEFT-MedSAM method uses only the lightweight mask decoder for training the model while keeping the pre-trained image encoder and prompt encoder frozen. The experiments performed on the ISIC 2018 benchmark dataset shows that PEFT-MedSAM obtains a dice coefficient of .9411 and an intersection over union value of .8918 when compared to both a fully trained U-Net baseline (.8715 dice coefficient) and zero-shot MedSAM inference (.8997 dice coefficient). The external validation of the model using PH2 dataset shows .9467 dice coefficient with +/- .0310 standard deviation. Supportive evidence for these claims include a p-value less than .0001 for Wilcoxon signed rank tests comparing the two datasets and bootstrap-estimated 95% confidence intervals of [.9364,.9447] that represent the estimated range of possible values for the average dice coefficient obtained by repeating the test. To increase clinical trustworthiness, we used Grad-CAM explainability along with a pointing game based evaluation methodology to evaluate the CNN baseline model on the validation set. The results showed that we had an accuracy rate of 98.27% on the validation set of 519 images and confirmed that the model classified regions containing skin lesions.

Problem

• skin lesion segmentation
• deep learning
• medical foundation models
• dermoscopic images
• melanoma detection

Innovation

• parameter-efficient fine-tuning
• Medical Segment Anything Model
• skin lesion segmentation
• explainable AI
• mask decoder