SUGFW+: An Uncertainty-guided Feature Weighting Framework for Cold Start Active Adaptation of SAM in Medical Image Segmentation

2026-08-17

AI Summary

This study addresses the inefficiency of sample selection and the disconnection from model training in Segment Anything Model (SAM)-based cold-start active learning for medical image segmentation. We propose SUGFW+, a framework that integrates patch-level feature weighting with global salient representation aggregation to enable an uncertainty-guided greedy selection strategy. By incorporating Uncertainty-Prompt Fine-Tuning (UPFT), SUGFW+ achieves deep coupling between sample selection and model optimization. Experiments on four public datasets demonstrate that SUGFW+ outperforms existing cold-start methods, significantly improving segmentation performance under low annotation budgets. These results confirm that the proposed approach effectively overcomes the limitations of traditional decoupled selection and training paradigms, offering a robust solution for resource-constrained medical imaging applications.

Abstract

Cold Start Active Learning (CSAL) is important in improving the performance of a medical image segmentation model with low annotation budget by querying a small subset for annotation from an unlabeled training set. Existing CSAL methods typically rely on inefficient dataset-specific Self-Supervised Learning (SSL) to map the unlabeled images into a feature space for sample selection. Recently, the advent of foundation models such as the Segment Anything Model (SAM) offer a promising alternative as the pre-trained model can provide strong generalizable feature embeddings, and allow high performance in downstream tasks after fine-tuning (adaptation). However, how to systematically exploit SAM's inherent embeddings for cold-start sample selection during adaptation with low annotation budget remains underexplored. To address this, we propose an extended SAM-based Uncertainty-guided Feature Weighting (SUGFW+) framework for CSAL and adaptation of SAM. Specifically, it leverages the SAM for Patch-level Feature and Uncertainty Calculation (PFUC), and introduces a Patch-based Global Distinct Representation (PGDR) module that aggregates patch-level embeddings into highly discriminative, uncertainty-aware image-level features. These features are then utilized by a Greedy Selection with Cluster and Uncertainty (GSCU) strategy to combine diversity and uncertainty during sample selection. Unlike prior CSAL methods that decouple sample selection from model training, SUGFW+ tightly integrates these two stages via an Uncertainty-Prompted Fine-Tuning (UPFT) process of SAM in model training. Extensive experiments on four public datasets demonstrate that SUGFW+ achieves state-of-the-art performance against existing CSAL methods. Code is available at https://github.com/HiLab-git/SUGFW-plus.

Problem

• Cold Start Active Learning
• Medical Image Segmentation
• Segment Anything Model
• Sample Selection
• Low Annotation Budget

Innovation

• Uncertainty-guided Feature Weighting
• Patch-based Global Distinct Representation
• Greedy Selection with Cluster and Uncertainty
• Uncertainty-Prompted Fine-Tuning
• Cold Start Active Learning