Multi-modal Integration Analysis of Alzheimer's Disease Using Large Language Models and Knowledge Graphs

2025-05-21

AI Summary

Alzheimer’s disease (AD) research faces challenges in integrating heterogeneous, unpaired, and cross-cohort multimodal data—including MRI, gene expression, biomarkers, EEG, and clinical metrics—due to their distributed nature and lack of subject-level alignment. Method: We propose the first large language model (LLM)-driven knowledge graph reasoning framework enabling population-level, ID-agnostic, concept-level cross-modal association mining and natural language hypothesis generation. Our approach integrates multimodal statistical feature selection, cross-cohort cross-validation, and expert consensus evaluation (Cohen’s κ = 0.82). Contribution/Results: We identify a novel pathological cascade—“metabolic risk → neuroinflammation → tau dysregulation”—and robust frontal EEG–gene expression associations (r = 0.42–0.58, p < 0.01; high-significance links: r > 0.6, p < 0.001), with effect sizes stable across cohorts (variance < 15%). These findings yield testable, mechanistically grounded hypotheses for AD pathogenesis and therapeutic target discovery.

Technology Category

• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data
• Machine Learning: Multimodal Learning
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

We propose a novel framework for integrating fragmented multi-modal data in Alzheimer's disease (AD) research using large language models (LLMs) and knowledge graphs. While traditional multimodal analysis requires matched patient IDs across datasets, our approach demonstrates population-level integration of MRI, gene expression, biomarkers, EEG, and clinical indicators from independent cohorts. Statistical analysis identified significant features in each modality, which were connected as nodes in a knowledge graph. LLMs then analyzed the graph to extract potential correlations and generate hypotheses in natural language. This approach revealed several novel relationships, including a potential pathway linking metabolic risk factors to tau protein abnormalities via neuroinflammation (r>0.6, p<0.001), and unexpected correlations between frontal EEG channels and specific gene expression profiles (r=0.42-0.58, p<0.01). Cross-validation with independent datasets confirmed the robustness of major findings, with consistent effect sizes across cohorts (variance<15%). The reproducibility of these findings was further supported by expert review (Cohen's k=0.82) and computational validation. Our framework enables cross modal integration at a conceptual level without requiring patient ID matching, offering new possibilities for understanding AD pathology through fragmented data reuse and generating testable hypotheses for future research.

Problem

• Integrating fragmented multi-modal Alzheimer's data using LLMs and knowledge graphs
• Enabling population-level analysis without matched patient IDs across datasets
• Discovering novel AD correlations and generating testable hypotheses

Innovation

• Integrates multi-modal data using LLMs and knowledge graphs
• Connects significant features as nodes in knowledge graph
• Generates hypotheses via LLM analysis of graph correlations