BrainPrompt: Multi-Level Brain Prompt Enhancement for Neurological Condition Identification

2025-04-12

AI Summary

Early diagnosis of neurodegenerative diseases (e.g., Alzheimer’s disease) using purely image-based models suffers from neglecting non-imaging priors and limited interpretability. To address this, we propose the Multi-level Brain Prompt Enhancement (MBPE) framework—the first to integrate knowledge-driven prompts generated by large language models (LLMs) into graph neural networks (GNNs), enabling semantic guidance at three hierarchical levels: brain regions (ROI-level), individual subjects (subject-level), and disease progression (disease-level). MBPE jointly leverages resting-state fMRI functional connectivity and domain-specific neuroscience knowledge to enhance both discriminative performance and model interpretability. Evaluated on two public fMRI datasets, MBPE significantly outperforms state-of-the-art methods. Biomarker analysis further validates that the learned features align with established neuroscientific priors. The source code is publicly available.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Humans and AI: Brain-Sensing and Analysis
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Neurological conditions, such as Alzheimer's Disease, are challenging to diagnose, particularly in the early stages where symptoms closely resemble healthy controls. Existing brain network analysis methods primarily focus on graph-based models that rely solely on imaging data, which may overlook important non-imaging factors and limit the model's predictive power and interpretability. In this paper, we present BrainPrompt, an innovative framework that enhances Graph Neural Networks (GNNs) by integrating Large Language Models (LLMs) with knowledge-driven prompts, enabling more effective capture of complex, non-imaging information and external knowledge for neurological disease identification. BrainPrompt integrates three types of knowledge-driven prompts: (1) ROI-level prompts to encode the identity and function of each brain region, (2) subject-level prompts that incorporate demographic information, and (3) disease-level prompts to capture the temporal progression of disease. By leveraging these multi-level prompts, BrainPrompt effectively harnesses knowledge-enhanced multi-modal information from LLMs, enhancing the model's capability to predict neurological disease stages and meanwhile offers more interpretable results. We evaluate BrainPrompt on two resting-state functional Magnetic Resonance Imaging (fMRI) datasets from neurological disorders, showing its superiority over state-of-the-art methods. Additionally, a biomarker study demonstrates the framework's ability to extract valuable and interpretable information aligned with domain knowledge in neuroscience. The code is available at https://github.com/AngusMonroe/BrainPrompt

Problem

• Diagnosing neurological conditions early is challenging due to symptom similarity with healthy controls.
• Existing brain network methods overlook non-imaging data, limiting predictive power and interpretability.
• BrainPrompt integrates multi-level knowledge prompts with GNNs and LLMs for improved disease identification.

Innovation

• Integrates LLMs with GNNs for brain analysis
• Uses multi-level knowledge-driven prompts
• Enhances interpretability and predictive power