Self-Attention Diffusion Models for Zero-Shot Biomedical Image Segmentation: Unlocking New Frontiers in Medical Imaging

2025-03-23

AI Summary

To address the scarcity of annotated data in medical image segmentation, this paper proposes ADZUS, a zero-shot segmentation framework that requires neither labeled data nor domain-specific priors. Methodologically, ADZUS deeply integrates self-attention mechanisms with pre-trained diffusion models via latent-space inversion and prompt-driven decoding, leveraging the generative and discriminative capabilities of diffusion models to achieve cross-modal, context-aware, and detail-sensitive unsupervised segmentation. Evaluated on skin lesion, chest X-ray infection, and white blood cell segmentation tasks, ADZUS achieves Dice scores of 88.7%–92.9% and IoU scores of 66.3%–93.3%, substantially outperforming existing zero-shot methods. Its core contributions are: (1) the first pure zero-shot medical image segmentation framework built entirely upon diffusion models; (2) a novel paradigm synergizing latent-space inversion with self-attention for holistic representation learning; and (3) complete elimination of fine-tuning and manual annotation dependencies, thereby advancing clinically deployable segmentation technologies.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Unsupervised & Self-Supervised Learning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Producing high-quality segmentation masks for medical images is a fundamental challenge in biomedical image analysis. Recent research has explored large-scale supervised training to enable segmentation across various medical imaging modalities and unsupervised training to facilitate segmentation without dense annotations. However, constructing a model capable of segmenting diverse medical images in a zero-shot manner without any annotations remains a significant hurdle. This paper introduces the Attention Diffusion Zero-shot Unsupervised System (ADZUS), a novel approach that leverages self-attention diffusion models for zero-shot biomedical image segmentation. ADZUS harnesses the intrinsic capabilities of pre-trained diffusion models, utilizing their generative and discriminative potentials to segment medical images without requiring annotated training data or prior domain-specific knowledge. The ADZUS architecture is detailed, with its integration of self-attention mechanisms that facilitate context-aware and detail-sensitive segmentations being highlighted. Experimental results across various medical imaging datasets, including skin lesion segmentation, chest X-ray infection segmentation, and white blood cell segmentation, reveal that ADZUS achieves state-of-the-art performance. Notably, ADZUS reached Dice scores ranging from 88.7% to 92.9% and IoU scores from 66.3% to 93.3% across different segmentation tasks, demonstrating significant improvements in handling novel, unseen medical imagery. It is noteworthy that while ADZUS demonstrates high effectiveness, it demands substantial computational resources and extended processing times. The model's efficacy in zero-shot settings underscores its potential to reduce reliance on costly annotations and seamlessly adapt to new medical imaging tasks, thereby expanding the diagnostic capabilities of AI-driven medical imaging technologies.

Problem

• Zero-shot biomedical image segmentation without annotations
• Leveraging self-attention diffusion models for medical imaging
• Reducing reliance on costly annotated training data

Innovation

• Self-attention diffusion models for zero-shot segmentation
• No annotated training data or domain knowledge needed
• State-of-the-art performance across diverse medical datasets