Prompting Foundation Models for Zero-Shot Ship Instance Segmentation in SAR Imagery

2026-04-20

AI Summary

This work addresses the challenge of ship instance segmentation in synthetic aperture radar (SAR) imagery, where pixel-level annotations are typically unavailable. The authors propose a zero-shot segmentation approach that leverages bounding boxes generated by a YOLOv11 detector as prompts to drive the Segment Anything Model 2 (SAM2) to produce instance masks—entirely without mask supervision. Notably, this method relies solely on a detector trained within the SAR domain to provide spatial constraints, effectively guiding a general-purpose vision foundation model to perform cross-modal segmentation without requiring fine-tuning or adapter modules, thereby substantially mitigating the domain gap between optical and SAR imagery. Evaluated on the SSDD benchmark, the proposed method achieves an average IoU of 0.637 (89% of the fully supervised baseline) and a ship detection rate of 89.2%, demonstrating its efficacy and scalability.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Synthetic Aperture Radar (SAR) plays a critical role in maritime surveillance, yet deep learning for SAR analysis is limited by the lack of pixel-level annotations. This paper explores how general-purpose vision foundation models can enable zero-shot ship instance segmentation in SAR imagery, eliminating the need for pixel-level supervision. A YOLOv11-based detector trained on open SAR datasets localizes ships via bounding boxes, which then prompt the Segment Anything Model 2 (SAM2) to produce instance masks without any mask annotations. Unlike prior SAM-based SAR approaches that rely on fine tuning or adapters, our method demonstrates that spatial constraints from a SAR-trained detector alone can effectively regularize foundation model predictions. This design partially mitigates the optical-SAR domain gap and enables downstream applications such as vessel classification, size estimation, and wake analysis. Experiments on the SSDD benchmark achieve a mean IoU of 0.637 (89% of a fully supervised baseline) with an overall ship detection rate of 89.2%, confirming a scalable, annotation-efficient pathway toward foundation-model-driven SAR image understanding.

Problem

• zero-shot
• ship instance segmentation
• SAR imagery
• pixel-level annotations
• domain gap

Innovation

• zero-shot segmentation
• foundation models
• SAR imagery
• prompting
• instance segmentation