Graph Adapter of EEG Foundation Models for Parameter Efficient Fine Tuning

2024-11-25

arXiv.org

AI Summary

To address the high computational cost of spatiotemporal modeling and severe scarcity of labeled data in EEG-based neurological disorder diagnosis, this paper proposes EG A, a lightweight graph adapter that integrates a graph neural network (GNN) into a pre-trained temporal backbone (BENDR) while freezing the backbone parameters and optimizing only a low-rank spatial relation modeling module. This is the first work to combine GNNs with parameter-efficient fine-tuning (PEFT) for explicit EEG sensor topology modeling, enabling synergistic spatiotemporal representation learning. Evaluated on major depressive disorder (MDD) classification and TUAB abnormality detection, EG A achieves up to a 16.1% improvement in F1-score, reduces GPU memory consumption by 42%, and matches full-parameter fine-tuning performance using only 10% of annotated data—significantly enhancing feasibility for low-resource clinical deployment.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

In diagnosing neurological disorders from electroencephalography (EEG) data, foundation models such as Transformers have been employed to capture temporal dynamics. Additionally, Graph Neural Networks (GNNs) are critical for representing the spatial relationships among EEG sensors. However, fine-tuning these large-scale models for both temporal and spatial features can be prohibitively large in computational cost, especially under the limited availability of labeled EEG datasets. We propose EEG-GraphAdapter (EGA), a parameter-efficient fine-tuning (PEFT) approach designed to address these challenges. EGA is integrated into a pre-trained temporal backbone model as a GNN-based module, freezing the backbone and allowing only the adapter to be fine-tuned. This enables the effective acquisition of EEG spatial representations, significantly reducing computational overhead and data requirements. Experimental evaluations on two healthcare-related downstream tasks-Major Depressive Disorder (MDD) and Abnormality Detection (TUAB)-show that EGA improves performance by up to 16.1% in F1-score compared with the backbone BENDR model, highlighting its potential for scalable and accurate EEG-based predictions.

Problem

• Efficient fine-tuning for EEG models
• Reducing computational cost in EEG analysis
• Enhancing spatial-temporal feature extraction in EEG

Innovation

• Graph Adapter integration
• Parameter-efficient fine-tuning
• Reduced computational overhead