SAMSA 2.0: Prompting Segment Anything with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation

2025-08-01

AI Summary

To address insufficient spectral information utilization and poor robustness under low-sample regimes and clinical noise in interactive segmentation of hyperspectral medical images, this paper proposes the Spectral Angle Prompting (SAP) mechanism. Without fine-tuning, SAP incorporates spectral similarity—quantified via spectral angle—as a prior prompt signal, early-fusing it into the spatial decoding pathway of the Segment Anything Model (SAM). By guiding spatial attention with spectral angle, SAP enables synergistic spectral–spatial multimodal modeling. Evaluated across multiple hyperspectral medical datasets, SAP achieves state-of-the-art Dice scores under zero-shot and few-shot settings—outperforming RGB-SAM by 3.8% and surpassing existing spectral fusion methods by 3.1%. Moreover, SAP significantly enhances model generalization and segmentation robustness in realistic, noise-corrupted clinical scenarios.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We present SAMSA 2.0, an interactive segmentation framework for hyperspectral medical imaging that introduces spectral angle prompting to guide the Segment Anything Model (SAM) using spectral similarity alongside spatial cues. This early fusion of spectral information enables more accurate and robust segmentation across diverse spectral datasets. Without retraining, SAMSA 2.0 achieves up to +3.8% higher Dice scores compared to RGB-only models and up to +3.1% over prior spectral fusion methods. Our approach enhances few-shot and zero-shot performance, demonstrating strong generalization in challenging low-data and noisy scenarios common in clinical imaging.

Problem

• Enhancing hyperspectral medical image segmentation accuracy
• Integrating spectral angles with spatial cues for SAM
• Improving few-shot and zero-shot segmentation performance

Innovation

• Spectral angle prompting enhances SAM
• Early fusion of spectral and spatial data
• Improves segmentation without model retraining