Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation

2026-06-18

AI Summary

High-resolution 3D CT generation remains challenging due to substantial computational costs and difficulties in preserving fine anatomical details. This work proposes PRDiT, a two-stage coarse-to-fine framework that first employs an MLP-based local denoiser on overlapping 3D patches to recover low-frequency structures and then leverages a global residual diffusion Transformer to model high-frequency details, thereby circumventing the bottlenecks inherent in autoencoder-based approaches. By integrating local blind denoising with a global residual diffusion mechanism and incorporating memory-efficient attention, PRDiT enables stable, scalable, end-to-end voxel-level synthesis. Experimental results on LIDC-IDRI and RAD-ChestCT demonstrate that PRDiT significantly outperforms HA-GAN, 3D LDM, and WDM-3D across multiple metrics, including 3D FID, MMD, and Wasserstein distance.

Abstract

Generating high-resolution 3D CT volumes with fine details remains challenging due to substantial computational demands and optimization difficulties inherent to existing generative models. In this paper, we propose the Pixel-Level Residual Diffusion Transformer (PRDiT), a scalable generative framework that synthesizes high-quality 3D medical volumes directly at voxel-level. PRDiT introduces a two-stage training architecture comprising 1) a local denoiser in the form of an MLP-based blind estimator operating on overlapping 3D patches to separate low-frequency structures efficiently, and 2) a global residual diffusion transformer employing memory-efficient attention to model and refine high-frequency residuals across entire volumes. This coarse-to-fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without the limitations of an autoencoder bottleneck. Extensive experiments conducted on the LIDC-IDRI and RAD-ChestCT datasets demonstrate that PRDiT consistently outperforms state-of-the-art models, such as HA-GAN, 3D LDM and WDM-3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.

Problem

• 3D CT volume generation
• high-resolution medical imaging
• generative models
• computational demands
• optimization difficulties

Innovation

• Pixel-Level Residual Diffusion Transformer
• 3D CT volume generation
• coarse-to-fine modeling
• memory-efficient attention
• voxel-level synthesis