SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis

2026-06-28

AI Summary

Fetal ultrasound images present significant challenges for existing vision-language models due to strong speckle noise, substantial acquisition variability, and ambiguous anatomical boundaries, hindering accurate modeling of clinically critical local structures. To address this, this work proposes SonoCLIP—the first mask-guided, region-controllable vision-language pretraining framework—which integrates segmentation masks as visual prompts into the encoder to enable joint global and local region contrastive learning. A sigmoid-based pairwise contrastive loss is further introduced to enhance training stability at scale. Pretrained on a newly curated multimodal fetal ultrasound dataset comprising over one million image-text pairs, SonoCLIP substantially outperforms current methods in cross-center zero-shot transfer tasks and supports both global and mask-guided inference, offering a clinically oriented foundation model for fetal ultrasound analysis.

Abstract

Vision-language foundation models have shown strong potential in medical image analysis. Although foundation models for ultrasound imaging have recently emerged, the domain remains particularly challenging due to severe speckle noise, acquisition variability, and subtle anatomical boundaries, leading to high inter-observer variability. Existing CLIP-based models rely primarily on global image-text alignment, limiting their sensitivity to clinically decisive local structures. We propose SonoCLIP, the first million-scale region-controllable fetal ultrasound vision-language foundation model that integrates segmentation masks as mask-channel visual prompts within the vision encoder, enabling joint global-local contrastive representation learning. To support scalable region-text alignment, we introduce a sigmoid-based pairwise contrastive loss that improves stability under large-scale supervision. We further curate a 1.44M-image multimodal fetal ultrasound dataset spanning 24 standard planes for large-scale pretraining. Extensive cross-center evaluations demonstrate that SonoCLIP achieves superior zero-shot transfer performance under both global and mask-guided inference, establishing a controllable and clinically oriented foundation model for fetal ultrasound analysis. Our code and data are available at https://github.com/Harrison-one/SonoCLIP.

Problem

• fetal ultrasound
• vision-language model
• local structure sensitivity
• speckle noise
• inter-observer variability

Innovation

• mask-guided vision-language pretraining
• region-aware contrastive learning
• fetal ultrasound foundation model
• segmentation mask prompting
• sigmoid-based pairwise contrastive loss