SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation

2025-07-31

AI Summary

To address data scarcity and cross-device hardware discrepancies in hyperspectral medical image segmentation, this paper proposes an interactive segmentation framework that synergistically integrates RGB foundation models with spectral analysis. The method introduces a band- and resolution-agnostic spectral feature fusion strategy, enabling synchronized guidance between RGB semantic segmentation and spectral angle mapper (SAM)-based similarity computation via user clicks. It further establishes a joint optimization mechanism unifying pre-trained RGB models, spectral metrics, and interactive learning, supporting few-shot and zero-shot adaptation. Evaluated on neurosurgical and porcine laparoscopic hyperspectral datasets, the framework achieves single-click Dice scores of 81.0% and 81.1%, respectively, improving to 93.4% and 89.2% with five clicks. Results demonstrate substantial gains in few-shot generalization and cross-platform robustness.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Hyperspectral imaging (HSI) provides rich spectral information for medical imaging, yet encounters significant challenges due to data limitations and hardware variations. We introduce SAMSA, a novel interactive segmentation framework that combines an RGB foundation model with spectral analysis. SAMSA efficiently utilizes user clicks to guide both RGB segmentation and spectral similarity computations. The method addresses key limitations in HSI segmentation through a unique spectral feature fusion strategy that operates independently of spectral band count and resolution. Performance evaluation on publicly available datasets has shown 81.0% 1-click and 93.4% 5-click DICE on a neurosurgical and 81.1% 1-click and 89.2% 5-click DICE on an intraoperative porcine hyperspectral dataset. Experimental results demonstrate SAMSA's effectiveness in few-shot and zero-shot learning scenarios and using minimal training examples. Our approach enables seamless integration of datasets with different spectral characteristics, providing a flexible framework for hyperspectral medical image analysis.

Problem

• Overcoming data limitations in hyperspectral medical imaging
• Integrating RGB and spectral data for accurate segmentation
• Enabling flexible analysis across varied spectral characteristics

Innovation

• Combines RGB model with spectral analysis
• Uses user clicks for segmentation guidance
• Fuses spectral features independently of band count