Is SAM3 ready for pathology segmentation?

2026-04-20

AI Summary

This study addresses the challenges of digital pathology image segmentation, which is hindered by high annotation costs and limited model generalization. To systematically evaluate the applicability of general-purpose segmentation models in this domain, the authors propose SAM3, the first comprehensive benchmarking protocol tailored for histopathology. Leveraging the Promptable Concept Segmentation framework that integrates textual and visual prompts, they conduct multi-scenario evaluations across datasets such as NuInsSeg, PanNuke, and GlaS under zero-shot, few-shot, and fully supervised settings at both tissue and nuclear levels. Their findings reveal significant performance gaps: textual prompts struggle to effectively activate nuclear concepts, visual prompts exhibit high sensitivity, and few-shot learning offers only marginal gains with poor robustness to noise. Task-specific adapters consistently outperform prompt-driven approaches, underscoring the need for domain-adapted solutions.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Is Segment Anything Model 3 (SAM3) capable in segmenting Any Pathology Images? Digital pathology segmentation spans tissue-level and nuclei-level scales, where traditional methods often suffer from high annotation costs and poor generalization. SAM3 introduces Promptable Concept Segmentation, offering a potential automated interface via text prompts. With this work, we propose a systematic evaluation protocol to explore the capability space of SAM3 in a structured manner. Specifically, we evaluate SAM3 under different supervision settings including zero-shot, few-shot, and supervised with varying prompting strategies. Our extensive evaluation on pathological datasets including NuInsSeg, PanNuke and GlaS, reveals that: 1.text-only prompts poorly activate nuclear concepts. 2.performance is highly sensitive to visual prompt types and budgets. 3.few-shot learning offers gains, but SAM3 lacks robustness against visual prompt noise. and 4.a significant gap persists between prompt-based usage and task-trained adapter-based reference. Our study delineates SAM3's boundaries in pathology image segmentation and provides practical guidance on the necessity of pathology domain adaptation.

Problem

• pathology segmentation
• Segment Anything Model
• prompt-based segmentation
• digital pathology
• model generalization

Innovation

• Promptable Concept Segmentation
• Pathology Image Segmentation
• Zero-shot Learning
• Few-shot Learning
• Visual Prompting