Multi-Modality Conditioned Variational U-Net for Field-of-View Extension in Brain Diffusion MRI

2024-09-20

arXiv.org

AI Summary

Incomplete dMRI field-of-view severely limits volumetric and tract-based whole-brain white matter connectome analysis. To address this, we propose a multimodal conditional variational U-Net framework that—uniquely—integrates T1-weighted anatomical priors and dMRI-derived local diffusion features through structured conditional modeling, rather than naïve concatenation, and incorporates diffusion tensor geometry constraints via angular coherence consistency (ACC) optimization to enhance physical plausibility. Evaluated on 96 cross-site subjects, our method significantly outperforms baselines (ACC improvement, *p* < 1e−5; tractogram Dice score increase, *p* < 0.01). The synthesized dMRI enables high-fidelity whole-brain tractography, effectively repairing fragmented fiber tracts and reducing tracking uncertainty in neurodegenerative analyses. Key innovations include: (i) structured conditional fusion of complementary multimodal information, and (ii) diffusion-geometry-guided generative modeling for biophysically informed dMRI synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

An incomplete field-of-view (FOV) in diffusion magnetic resonance imaging (dMRI) can severely hinder the volumetric and bundle analyses of whole-brain white matter connectivity. Although existing works have investigated imputing the missing regions using deep generative models, it remains unclear how to specifically utilize additional information from paired multi-modality data and whether this can enhance the imputation quality and be useful for downstream tractography. To fill this gap, we propose a novel framework for imputing dMRI scans in the incomplete part of the FOV by integrating the learned diffusion features in the acquired part of the FOV to the complete brain anatomical structure. We hypothesize that by this design the proposed framework can enhance the imputation performance of the dMRI scans and therefore be useful for repairing whole-brain tractography in corrupted dMRI scans with incomplete FOV. We tested our framework on two cohorts from different sites with a total of 96 subjects and compared it with a baseline imputation method that treats the information from T1w and dMRI scans equally. The proposed framework achieved significant improvements in imputation performance, as demonstrated by angular correlation coefficient (p<1E-5), and in downstream tractography accuracy, as demonstrated by Dice score (p<0.01). Results suggest that the proposed framework improved imputation performance in dMRI scans by specifically utilizing additional information from paired multi-modality data, compared with the baseline method. The imputation achieved by the proposed framework enhances whole brain tractography, and therefore reduces the uncertainty when analyzing bundles associated with neurodegenerative.

Problem

• Extend incomplete FOV in brain dMRI using multi-modality data
• Enhance dMRI imputation quality for accurate tractography analysis
• Utilize paired T1w and dMRI data to improve white matter connectivity

Innovation

• Multi-modality conditioned variational U-Net
• Integrates diffusion features with anatomy
• Enhances dMRI imputation and tractography