2D Versus 3D Diffusion for In Silico Training of Interventional X-ray AI Models

2026-06-19

AI Summary

This study addresses the scarcity of annotated real-world X-ray data for training AI models in interventional radiology, a challenge exacerbated by the difficulty of obtaining high-quality 3D anatomical models required by existing CT-based digitally reconstructed radiograph (DRR) synthesis methods. The work presents the first systematic comparison between 2D and 3D diffusion models for synthetic X-ray generation: one approach employs a 3D conditional latent diffusion model to synthesize CT volumes for DRR creation, while the other directly generates X-ray images using a view-conditioned 2D diffusion model, both aimed at training anatomical landmark detection models. Experimental results demonstrate that models trained solely on 2D diffusion–synthesized data achieve performance on real X-rays comparable to those trained on real data, confirming the feasibility of generating effective training data without access to actual 3D anatomical structures and thereby overcoming a critical data acquisition bottleneck.

Abstract

The ability to synthesize realistic X-ray images has catalyzed the development of AI models for X-ray image-guided procedures, which otherwise suffer from a lack of available annotated data. Prior work has demonstrated the effectiveness of mechanistic simulation of digitally reconstructed radiographs (DRRs) as a training data source for a myriad of tasks, including segmentation and anatomical landmark detection, with comparable or superior performance to real data training. However, mechanistic DRR synthesis still relies on the availability of annotated high-resolution anatomical models. Deriving these from CT images of real patients or specimens imposes an undesirable bottleneck on data quantity and variability. In this work, we explore two methods for synthesizing training data: (1) a 3D conditional latent diffusion model that generates CT volumes to use as inputs for mechanistic DRR generation without real, 3D anatomical models, and (2) a view-conditioned 2D diffusion model that produces synthetic X-rays. In controlled experiments, we demonstrate that synthetic 2D diffusion-based X-rays can be used to train an anatomical landmark detection model that generalized to real X-ray images with performance rivaling that of a model trained on real X-ray images. Thus, we provide preliminary evidence that synthetic, 2D diffusion-based training data can substitute for real X-ray data, identifying a promising avenue towards generating large, diverse datasets for training robust AI models in interventional X-ray imaging.

Problem

• X-ray imaging
• data scarcity
• anatomical models
• synthetic data
• interventional AI

Innovation

• 2D diffusion
• synthetic X-ray
• anatomical landmark detection
• data synthesis
• interventional imaging