PGE-SAM: Prompt-Guided Feature Enhancement for Interactive Segmentation under Degradation

2026-06-29

AI Summary

This work addresses the significant performance degradation of the Segment Anything Model (SAM) in interactive segmentation on images affected by noise, blur, and compression artifacts. To mitigate this issue, the authors propose a lightweight enhancement method that explicitly integrates user prompts into the feature enhancement process for the first time. They design a prompt-guided generator to spatially focus on restoring regions of interest, coupled with multi-scale feature interaction and a segmentation-oriented foreground reconstruction loss. The approach achieves state-of-the-art robustness across varying levels of image degradation in both medical and natural images while preserving generalization on clean inputs, adding fewer than one-fifth the parameters of existing methods. Additionally, the study introduces DM-Seg, the first benchmark for interactive segmentation on degraded medical images.

Abstract

Segment Anything Model (SAM) has revolutionized promptable image segmentation with strong zero-shot generalization. However, its performance degrades substantially under real-world imaging artifacts such as noise, blur, and compression. Existing methods restore features globally without focusing on segmentation-relevant regions and neglect SAM's iterative refinement mechanism, leading to suboptimal performance in interactive settings. We propose Prompt-Guided Feature Enhancement SAM (PGE-SAM), a framework that explicitly leverages user prompts and prior mask predictions to spatially guide the feature restoration process toward regions of interest through a Prompt Guidance Generator. To recover fine-grained details lost under degradation, we introduce Multi-Scale Features Interaction to incorporate low-level encoder features, along with a Foreground Reconstruction Loss that restricts feature-level supervision to the segmentation target. Furthermore, we present DM-Seg, a benchmark for interactive segmentation on degraded medical images, spanning multiple imaging modalities with both general and modality-specific degradations at varying severity levels. Extensive experiments demonstrate that PGE-SAM achieves SOTA robustness on both medical and natural image domains across multiple degradation levels, while maintaining generalization to clean images and adding less than one-fifth of the parameters of prior methods.

Problem

• image degradation
• interactive segmentation
• feature enhancement
• Segment Anything Model
• promptable segmentation

Innovation

• Prompt-Guided Feature Enhancement
• Interactive Segmentation
• Image Degradation
• Multi-Scale Features Interaction
• Foreground Reconstruction Loss