Bridging Brain Connectomes and Clinical Reports for Early Alzheimer's Disease Diagnosis

2025-08-06

AI Summary

This study addresses the challenge of establishing cross-modal associations between neuroimaging data and clinical text reports to improve the accuracy and timeliness of early Alzheimer’s disease (AD) diagnosis. We propose the first cross-modal framework that models brain functional subnetworks as semantic “imaging tokens” and aligns them with clinical text tokens in a shared latent space. The method integrates graph neural networks—designed to extract topological features from connectomes—with pretrained language models—used to encode clinical notes—enabling interpretable, network-level semantic mapping between neuroimaging and physician observations. Evaluated on the ADNI dataset, our approach achieves state-of-the-art performance in mild cognitive impairment identification. Furthermore, it uncovers biologically and clinically meaningful connectome–text association patterns, offering a novel paradigm for multimodal biomarker discovery.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Integrating brain imaging data with clinical reports offers a valuable opportunity to leverage complementary multimodal information for more effective and timely diagnosis in practical clinical settings. This approach has gained significant attention in brain disorder research, yet a key challenge remains: how to effectively link objective imaging data with subjective text-based reports, such as doctors' notes. In this work, we propose a novel framework that aligns brain connectomes with clinical reports in a shared cross-modal latent space at both the subject and connectome levels, thereby enhancing representation learning. The key innovation of our approach is that we treat brain subnetworks as tokens of imaging data, rather than raw image patches, to align with word tokens in clinical reports. This enables a more efficient identification of system-level associations between neuroimaging findings and clinical observations, which is critical since brain disorders often manifest as network-level abnormalities rather than isolated regional alterations. We applied our method to mild cognitive impairment (MCI) using the ADNI dataset. Our approach not only achieves state-of-the-art predictive performance but also identifies clinically meaningful connectome-text pairs, offering new insights into the early mechanisms of Alzheimer's disease and supporting the development of clinically useful multimodal biomarkers.

Problem

• Align brain connectomes with clinical reports for early Alzheimer's diagnosis
• Link objective imaging data to subjective clinical text effectively
• Identify system-level associations between neuroimaging and clinical observations

Innovation

• Aligns brain connectomes with clinical reports
• Treats brain subnetworks as imaging tokens
• Identifies system-level neuroimaging-clinical associations