X-ray2CTPA: Generating 3D CTPA scans from 2D X-ray conditioning

2024-06-23

arXiv.org

AI Summary

This study addresses the high radiation dose, cost, and limited accessibility of conventional computed tomography pulmonary angiography (CTPA). We propose the first end-to-end, cross-modal, X-ray-to-3D-CTPA generation method based on a conditional diffusion model. By integrating anatomical prior modeling with a radiologist-driven evaluation framework, our approach synthesizes diagnostic-quality, high-resolution, contrast-enhanced 3D pulmonary arterial CT volumes directly from a single low-contrast 2D chest radiograph. The generated images achieve clinical validation by expert radiologists and significantly improve pulmonary embolism classification performance (notably increasing AUC). To our knowledge, this is the first work to apply diffusion models to 3D volumetric cross-modal medical image synthesis. It establishes a novel paradigm for low-cost, low-radiation alternative imaging diagnostics and advances the technical frontier of multimodal medical image translation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Chest X-rays or chest radiography (CXR), commonly used for medical diagnostics, typically enables limited imaging compared to computed tomography (CT) scans, which offer more detailed and accurate three-dimensional data, particularly contrast-enhanced scans like CT Pulmonary Angiography (CTPA). However, CT scans entail higher costs, greater radiation exposure, and are less accessible than CXRs. In this work we explore cross-modal translation from a 2D low contrast-resolution X-ray input to a 3D high contrast and spatial-resolution CTPA scan. Driven by recent advances in generative AI, we introduce a novel diffusion-based approach to this task. We evaluate the models performance using both quantitative metrics and qualitative feedback from radiologists, ensuring diagnostic relevance of the generated images. Furthermore, we employ the synthesized 3D images in a classification framework and show improved AUC in a PE categorization task, using the initial CXR input. The proposed method is generalizable and capable of performing additional cross-modality translations in medical imaging. It may pave the way for more accessible and cost-effective advanced diagnostic tools. The code for this project is available: https://github.com/NoaCahan/X-ray2CTPA .

Problem

• Convert 2D X-rays to 3D CTPA scans using diffusion models
• Improve pulmonary embolism classification with synthetic 3D images
• Enable cost-effective diagnostic tools via cross-modal translation

Innovation

• Diffusion models for 2D to 3D medical image translation
• Enhanced PE classification using synthesized CTPA scans
• Generalizable cross-modality translation in medical imaging