Trustworthy Enhanced Multi-view Multi-modal Alzheimer’s Disease Prediction with Brain-wide Imaging Transcriptomics Data

2024-06-21

IEEE International Conference on Bioinformatics and Biomedicine

AI Summary

Existing Alzheimer’s disease (AD) prediction methods predominantly rely on neuroimaging and genetic data, overlooking transcriptional foundations and inter-modal heterogeneity. To address this, we propose TMM—a trustworthy multi-view, multimodal graph attention framework—that integrates whole-brain transcriptomic data (Allen Human Brain Atlas, AHBA) with AV45-PET, FDG-PET, and VBM-MRI for the first time. TMM constructs region-specific RRI (regional RNA–imaging) co-functional networks and introduces a dual-perspective modeling mechanism coupled with a True–False Collaborative Probability calibration strategy (TFCP) to achieve cross-modal adaptive confidence weighting. Evaluated on the ADNI dataset, TMM achieves an average 4.2% improvement in three-class classification accuracy (AD/EMCI/LMCI) and an AUC of 0.923, significantly outperforming state-of-the-art methods. The source code and preprocessed data are publicly released.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Brain transcriptomics provides insights into the molecular mechanisms by which the brain coordinates its functions and processes. However, existing multimodal methods for predicting Alzheimer’s disease (AD) primarily rely on imaging and sometimes genetic data, often neglecting the transcriptomic basis of brain. Furthermore, while striving to integrate complementary information between modalities, most studies overlook the informativeness disparities between modalities. Here, we propose TMM, a trusted multiview multimodal graph attention framework for AD diagnosis, using extensive brain-wide transcriptomics and imaging data. First, we construct view-specific brain regional co-function networks (RRIs) from transcriptomics and multimodal radiomics data to incorporate interaction information from both biomolecular and imaging perspectives. Next, we apply graph attention (GAT) processing to each RRI network to produce graph embeddings and employ cross-modal attention to fuse transcriptomics-derived embedding with each imaging-derived embedding. Finally, a novel true-false-harmonized class probability (TFCP) strategy is designed to assess and adaptively adjust the prediction confidence of each modality for AD diagnosis. We evaluate TMM using the AHBA database with brain-wide transcriptomics data and the ADNI database with three imaging modalities (AV45-PET, FDG-PET, and VBM-MRI). The results demonstrate the superiority of our method in identifying AD, EMCI, and LMCI compared to state-of-the-arts. Code and data are available at https://github.com/Yaolab-fantastic/TMM.

Problem

• Incorporates brain-wide transcriptomics data often neglected in Alzheimer's prediction
• Addresses informativeness disparities between imaging and transcriptomics modalities
• Proposes a trusted multimodal framework for accurate Alzheimer's disease diagnosis

Innovation

• Constructs brain regional co-function networks from transcriptomics and imaging
• Uses graph attention and cross-modal attention for fusion
• Implements true-false-harmonized class probability for confidence adjustment