TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model

2026-07-15

AI Summary

This work addresses the challenges of high memory consumption and limited reconstruction and generation quality in high-resolution 3D medical image synthesis by proposing an efficient generative framework based on triplane representation. The method employs a decoder-only autoencoder to learn compact triplane features and introduces a triplane-aware cross-attention diffusion mechanism to enable effective cross-plane feature fusion and high-fidelity generation. By leveraging a pretrained decoder, the approach substantially reduces GPU memory usage and accelerates volumetric reconstruction. Evaluated on the BrainTumour, Pancreas, and Colon datasets, the model consistently outperforms existing methods across multiple metrics—including MSE, SSIM, and Wasserstein GAN-based evaluations—demonstrating its effectiveness and strong generalization capability.

Abstract

We introduce TCAM-Diff, a novel 3D medical image generation model that reduces the memory requirements to encode and generate high-resolution 3D data. This model utilizes a decoder-only autoencoder method to learn triplane representation from dense volume and leverages generalization operations to prevent overfitting. Subsequently, it uses a triplane-aware cross-attention diffusion model to learn and integrate these features effectively. Furthermore, the features generated by the diffusion model can be rapidly transformed into 3D volumes using a pre-trained decoder module. Our experiments on three different scales of medical datasets, BrainTumour 128 x 128 x 128, Pancreas 256 x 256 x 256, and Colon 512 x 512 x 512, demonstrate outstanding results. We utilized MSE and SSIM to assess reconstruction quality and leveraged the Wasserstein Generative Adversarial Network (W-GAN) critic to assess generative quality. Comparisons with existing approaches show that our method gives better reconstruction and generation results than other encoder-decoder methods with similar-sized latent spaces.

Problem

• 3D medical image generation
• high-resolution
• memory efficiency
• image reconstruction
• generative modeling

Innovation

• triplane representation
• cross-attention diffusion
• decoder-only autoencoder
• 3D medical image generation
• memory-efficient generative modeling