GFE-Mamba: Mamba-based AD Multi-modal Progression Assessment via Generative Feature Extraction from MCI

2024-07-22

arXiv.org

AI Summary

To address the challenges of PET data scarcity and heterogeneous multimodal fusion in predicting progression from mild cognitive impairment (MCI) to Alzheimer’s disease (AD), this paper proposes a novel multimodal learning framework. First, a generative feature extractor (GFE) synthesizes high-fidelity PET features to mitigate the absence of real PET scans. Second, a Mamba-based backbone augmented with pixel-wise bidirectional cross-attention (Bi-cross Attention) enables fine-grained MRI/PET alignment and long-range sequence modeling. Third, generative feature distillation and multimodal feature fusion are integrated to enhance representation consistency and discriminability. Evaluated on MCI-to-AD progression prediction, our method achieves significant improvements over existing state-of-the-art approaches, empirically validating the efficacy of synthesized PET features. The code and pretrained weights are publicly released to foster reproducibility and further research.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Alzheimer's Disease (AD) is a progressive, irreversible neurodegenerative disorder that often originates from Mild Cognitive Impairment (MCI). This progression results in significant memory loss and severely affects patients' quality of life. Clinical trials have consistently shown that early and targeted interventions for individuals with MCI may slow or even prevent the advancement of AD. Research indicates that accurate medical classification requires diverse multimodal data, including detailed assessment scales and neuroimaging techniques like Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET). However, simultaneously collecting the aforementioned three modalities for training presents substantial challenges. To tackle these difficulties, we propose GFE-Mamba, a multimodal classifier founded on Generative Feature Extractor. The intermediate features provided by this Extractor can compensate for the shortcomings of PET and achieve profound multimodal fusion in the classifier. The Mamba block, as the backbone of the classifier, enables it to efficiently extract information from long-sequence scale information. Pixel-level Bi-cross Attention supplements pixel-level information from MRI and PET. We provide our rationale for developing this cross-temporal progression prediction dataset and the pre-trained Extractor weights. Our experimental findings reveal that the GFE-Mamba model effectively predicts the progression from MCI to AD and surpasses several leading methods in the field. Our source code is available at https://github.com/Tinysqua/GFE-Mamba.

Problem

• Mild Cognitive Impairment (MCI)
• Alzheimer's Disease (AD)
• Feature Extraction

Innovation

• GFE-Mamba
• MCI-to-AD prediction
• Medical imaging analysis