Prompt-DAS: Annotation-Efficient Prompt Learning for Domain Adaptive Semantic Segmentation of Electron Microscopy Images

2025-09-23

AI Summary

To address the high annotation cost and poor cross-domain generalization in organelle segmentation for large-scale electron microscopy (EM) images, this paper proposes Prompt-DAS: a point-prompt-based multi-task domain adaptive semantic segmentation framework. Prompt-DAS jointly optimizes the primary segmentation task and an auxiliary center-point detection task, while incorporating a prompt-guided contrastive learning mechanism to enable flexible support for arbitrary numbers of point prompts during both training and inference. Crucially, it operates without instance-level annotations or dense pixel-wise masks, unifying unsupervised domain adaptation (UDA), weakly supervised domain adaptation (WDA), and interactive segmentation under a single framework. Evaluated on multiple EM benchmarks, Prompt-DAS consistently outperforms state-of-the-art UDA, WDA, and SAM-based methods, achieving superior trade-offs between segmentation accuracy and annotation efficiency.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Segmentation
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Domain adaptive segmentation (DAS) of numerous organelle instances from large-scale electron microscopy (EM) is a promising way to enable annotation-efficient learning. Inspired by SAM, we propose a promptable multitask framework, namely Prompt-DAS, which is flexible enough to utilize any number of point prompts during the adaptation training stage and testing stage. Thus, with varying prompt configurations, Prompt-DAS can perform unsupervised domain adaptation (UDA) and weakly supervised domain adaptation (WDA), as well as interactive segmentation during testing. Unlike the foundation model SAM, which necessitates a prompt for each individual object instance, Prompt-DAS is only trained on a small dataset and can utilize full points on all instances, sparse points on partial instances, or even no points at all, facilitated by the incorporation of an auxiliary center-point detection task. Moreover, a novel prompt-guided contrastive learning is proposed to enhance discriminative feature learning. Comprehensive experiments conducted on challenging benchmarks demonstrate the effectiveness of the proposed approach over existing UDA, WDA, and SAM-based approaches.

Problem

• Enables annotation-efficient semantic segmentation of electron microscopy images
• Performs domain adaptation using minimal prompts for organelle instance segmentation
• Addresses limitations of foundation models by requiring fewer training prompts

Innovation

• Promptable multitask framework for domain adaptation
• Auxiliary center-point detection enabling flexible prompting
• Prompt-guided contrastive learning enhances feature discrimination