NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentation

2024-08-21

Knowledge-Based Systems

AI Summary

To address weak cross-domain generalization and reliance on manually annotated bounding boxes in nuclear segmentation, this paper proposes SAM-Cell: an end-to-end cross-domain instance segmentation framework tailored for medical images. Methodologically, we introduce the Heterogeneous Space Adapter (HS-Adapter) to enhance SAM’s representation capability across staining and tissue domains; design a Gaussian Kernel Prompt Encoder (GKP-Encoder) that generates density-guided masks from single-point prompts; and develop a Two-Stage Mask Decoder (TSM-Decoder) enabling joint semantic-to-instance optimization, augmented with a domain-invariant feature alignment mechanism. Evaluated on multiple unseen staining and tissue domains, SAM-Cell achieves state-of-the-art performance, significantly improving cross-domain generalization while eliminating manual bounding box annotation and post-hoc morphological refinement. The code is publicly available.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Security

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Security and Privacy: Data transparency and provenance

Abstract

Domain-generalized nuclei segmentation refers to the generalizability of models to unseen domains based on knowledge learned from source domains and is challenged by various image conditions, cell types, and stain strategies. Recently, the Segment Anything Model (SAM) has made great success in universal image segmentation by interactive prompt modes (e.g., point and box). Despite its strengths, the original SAM presents limited adaptation to medical images. Moreover, SAM requires providing manual bounding box prompts for each object to produce satisfactory segmentation masks, so it is laborious in nuclei segmentation scenarios. To address these limitations, we propose a domain-generalizable framework for nuclei image segmentation, abbreviated to NuSegDG. Specifically, we first devise a Heterogeneous Space Adapter (HS-Adapter) to learn multi-dimensional feature representations of different nuclei domains by injecting a small number of trainable parameters into the image encoder of SAM. To alleviate the labor-intensive requirement of manual prompts, we introduce a Gaussian-Kernel Prompt Encoder (GKP-Encoder) to generate density maps driven by a single point, which guides segmentation predictions by mixing position prompts and semantic prompts. Furthermore, we present a Two-Stage Mask Decoder (TSM-Decoder) to effectively convert semantic masks to instance maps without the manual demand for morphological shape refinement. Based on our experimental evaluations, the proposed NuSegDG demonstrates state-of-the-art performance in nuclei instance segmentation, exhibiting superior domain generalization capabilities. The source code is available at https://github.com/xq141839/NuSegDG.

Problem

• Improving domain-generalized nuclei segmentation across diverse image conditions
• Reducing manual prompt dependency in SAM for medical image segmentation
• Enhancing instance segmentation accuracy without manual shape refinement

Innovation

• HS-Adapter for multi-dimensional feature learning
• GKP-Encoder for automatic prompt generation
• TSM-Decoder for instance mask conversion