Parameter-Efficient Fine-Tuning of 3D DDPM for MRI Image Generation Using Tensor Networks

2025-07-24

AI Summary

Parameter-efficient fine-tuning (PEFT) of 3D U-Net diffusion models for MRI synthesis remains challenging, particularly due to insufficient investigation into low-parameter representations of 3D convolutions. Method: We propose TenVOO—the first PEFT framework incorporating tensor networks into 3D diffusion models—by modeling low-rank 3D convolutional kernels via tensor voxel operators, thereby capturing complex spatial dependencies under extremely tight parameter budgets. TenVOO synergistically integrates tensor decomposition with low-rank tensor networks to enable few-shot, low-overhead adaptation of pre-trained 3D DDPM-U-Nets. Results: On ADNI, PPMI, and BraTS2021, TenVOO achieves state-of-the-art MS-SSIM performance using only 0.3% of the original model’s trainable parameters, significantly enhancing both spatial modeling fidelity and training efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We address the challenge of parameter-efficient fine-tuning (PEFT) for three-dimensional (3D) U-Net-based denoising diffusion probabilistic models (DDPMs) in magnetic resonance imaging (MRI) image generation. Despite its practical significance, research on parameter-efficient representations of 3D convolution operations remains limited. To bridge this gap, we propose Tensor Volumetric Operator (TenVOO), a novel PEFT method specifically designed for fine-tuning DDPMs with 3D convolutional backbones. Leveraging tensor network modeling, TenVOO represents 3D convolution kernels with lower-dimensional tensors, effectively capturing complex spatial dependencies during fine-tuning with few parameters. We evaluate TenVOO on three downstream brain MRI datasets-ADNI, PPMI, and BraTS2021-by fine-tuning a DDPM pretrained on 59,830 T1-weighted brain MRI scans from the UK Biobank. Our results demonstrate that TenVOO achieves state-of-the-art performance in multi-scale structural similarity index measure (MS-SSIM), outperforming existing approaches in capturing spatial dependencies while requiring only 0.3% of the trainable parameters of the original model. Our code is available at: https://github.com/xiaovhua/tenvoo

Problem

• Efficient fine-tuning of 3D DDPMs for MRI generation
• Reducing parameters in 3D convolution operations
• Capturing spatial dependencies with minimal trainable parameters

Innovation

• Tensor Volumetric Operator for 3D DDPM fine-tuning
• Leverages tensor networks for parameter efficiency
• Captures spatial dependencies with minimal parameters