Ultrasound Image Generation using Latent Diffusion Models

2025-02-12

AI Summary

This work addresses the critical scarcity of ultrasound images for rare diseases in medical imaging. We propose the first latent diffusion model (LDM) framework specifically designed for ultrasound generation. Methodologically, we pioneer the adaptation of Stable Diffusion to the ultrasound domain via multi-stage fine-tuning—including training on the BUSI dataset and clinical prompt engineering—augmented by ControlNet-guided synthesis using segmentation maps to ensure precise anatomical structure and pathological feature control. Innovatively, we introduce an anatomical consistency constraint to enhance clinical plausibility. Generated images were evaluated in a double-blind assessment by three ultrasound specialists and one radiologist, all of whom consistently rated them as realistic and clinically credible. The source code and pretrained models are publicly released to facilitate rapid, community-driven construction of high-quality breast ultrasound datasets.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Consent frameworks and practices on the web

Abstract

Diffusion models for image generation have been a subject of increasing interest due to their ability to generate diverse, high-quality images. Image generation has immense potential in medical imaging because open-source medical images are difficult to obtain compared to natural images, especially for rare conditions. The generated images can be used later to train classification and segmentation models. In this paper, we propose simulating realistic ultrasound (US) images by successive fine-tuning of large diffusion models on different publicly available databases. To do so, we fine-tuned Stable Diffusion, a state-of-the-art latent diffusion model, on BUSI (Breast US Images) an ultrasound breast image dataset. We successfully generated high-quality US images of the breast using simple prompts that specify the organ and pathology, which appeared realistic to three experienced US scientists and a US radiologist. Additionally, we provided user control by conditioning the model with segmentations through ControlNet. We will release the source code at http://code.sonography.ai/ to allow fast US image generation to the scientific community.

Problem

• Generate realistic ultrasound images
• Fine-tune diffusion models on medical datasets
• Enhance training for classification and segmentation models

Innovation

• Latent Diffusion Models
• Fine-tuning on BUSI
• ControlNet for segmentation