A Multimodal Deep Learning Approach for White Matter Shape Prediction in Diffusion MRI Tractography

2025-04-25

AI Summary

Conventional white matter tract shape measurement is computationally expensive and poorly scalable to large-scale neuroimaging datasets. Method: We propose Tract2Shape—the first multimodal deep learning framework integrating point-cloud geometric structure with scalar tabular features to efficiently predict ten white matter morphometric metrics. Our architecture jointly optimizes a novel point-cloud neural network and a tabular encoder, augmented by PCA-based dimensionality reduction that predicts the top five principal components—enhancing inference efficiency without sacrificing accuracy. Contribution/Results: On the HCP-YA dataset, Tract2Shape achieves state-of-the-art performance, attaining the highest average Pearson correlation coefficient (r) and lowest normalized mean squared error (nMSE). Crucially, it demonstrates strong cross-dataset generalizability: when transferred to the PPMI cohort, it maintains high predictive fidelity (r > 0.85, nMSE < 0.12), confirming its robustness and scalability for large-scale neuroimaging analysis.

Technology Category

• Machine Learning: Hardware-aware ML
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Computer Vision: Multi-modal Vision

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

Shape measures have emerged as promising descriptors of white matter tractography, offering complementary insights into anatomical variability and associations with cognitive and clinical phenotypes. However, conventional methods for computing shape measures are computationally expensive and time-consuming for large-scale datasets due to reliance on voxel-based representations. We propose Tract2Shape, a novel multimodal deep learning framework that leverages geometric (point cloud) and scalar (tabular) features to predict ten white matter tractography shape measures. To enhance model efficiency, we utilize a dimensionality reduction algorithm for the model to predict five primary shape components. The model is trained and evaluated on two independently acquired datasets, the HCP-YA dataset, and the PPMI dataset. We evaluate the performance of Tract2Shape by training and testing it on the HCP-YA dataset and comparing the results with state-of-the-art models. To further assess its robustness and generalization ability, we also test Tract2Shape on the unseen PPMI dataset. Tract2Shape outperforms SOTA deep learning models across all ten shape measures, achieving the highest average Pearson's r and the lowest nMSE on the HCP-YA dataset. The ablation study shows that both multimodal input and PCA contribute to performance gains. On the unseen testing PPMI dataset, Tract2Shape maintains a high Pearson's r and low nMSE, demonstrating strong generalizability in cross-dataset evaluation. Tract2Shape enables fast, accurate, and generalizable prediction of white matter shape measures from tractography data, supporting scalable analysis across datasets. This framework lays a promising foundation for future large-scale white matter shape analysis.

Problem

• Predicts white matter tractography shape measures efficiently
• Reduces computational cost for large-scale MRI datasets
• Enhances accuracy and generalizability in cross-dataset evaluation

Innovation

• Multimodal deep learning for white matter shape prediction
• Combines geometric and scalar features efficiently
• Uses PCA for dimensionality reduction and performance