Graph Conditioned Diffusion for Controllable Histopathology Image Generation

2025-10-08

AI Summary

Existing diffusion models for medical image generation operate in unstructured noise spaces, hindering fine-grained, semantically consistent control over anatomical structures in pathological images. Method: We propose a graph-conditioned diffusion generative framework that, for the first time, encodes spatial relationships and morphological characteristics of tissue structures as graph-structured data—serving as a semantic prior injected into the diffusion process. By integrating graph neural networks with Transformer modules, our method enables object-level, fine-grained manipulation within the noisy latent space. Contribution/Results: The framework significantly improves anatomical plausibility and interpretability of generated images. Evaluated on real pathological image synthesis, the generated data effectively substitutes real annotated samples for downstream segmentation tasks, achieving performance on par with models trained on ground-truth data. This establishes a novel paradigm for few-shot medical image generation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

Recent advances in Diffusion Probabilistic Models (DPMs) have set new standards in high-quality image synthesis. Yet, controlled generation remains challenging, particularly in sensitive areas such as medical imaging. Medical images feature inherent structure such as consistent spatial arrangement, shape or texture, all of which are critical for diagnosis. However, existing DPMs operate in noisy latent spaces that lack semantic structure and strong priors, making it difficult to ensure meaningful control over generated content. To address this, we propose graph-based object-level representations for Graph-Conditioned-Diffusion. Our approach generates graph nodes corresponding to each major structure in the image, encapsulating their individual features and relationships. These graph representations are processed by a transformer module and integrated into a diffusion model via the text-conditioning mechanism, enabling fine-grained control over generation. We evaluate this approach using a real-world histopathology use case, demonstrating that our generated data can reliably substitute for annotated patient data in downstream segmentation tasks. The code is available here.

Problem

• Generating histopathology images with controlled structural features
• Addressing semantic structure limitations in diffusion models
• Enabling fine-grained control over medical image generation

Innovation

• Graph-based object-level representations for histopathology generation
• Graph nodes encode structure features and relationships
• Transformer-processed graphs integrated via text-conditioning mechanism