AdaBrain-Bench: Benchmarking Brain Foundation Models for Brain-Computer Interface Applications

2025-07-13

AI Summary

The non-invasive brain–computer interface (BCI) field lacks a comprehensive, scalable evaluation benchmark tailored for brain foundation models, hindering their practical deployment. Method: We introduce AdaBrain-Bench—the first large-scale, standardized benchmark specifically designed for brain foundation models. It covers seven canonical BCI tasks and supports diverse transfer learning scenarios, including cross-subject, multi-subject, and few-shot evaluation. The benchmark features a unified task adaptation pipeline, multidimensional performance metrics (e.g., generalizability, robustness, data efficiency), and an open-source toolchain for systematic, reproducible EEG decoding model assessment. Results: Extensive experiments on multiple publicly available brain foundation models reveal their performance boundaries across tasks and data regimes, yielding actionable guidance for model selection. Both the benchmark and codebase are open-sourced to advance standardization and reproducibility in brain-AI research.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Humans and AI: Brain-Sensing and Analysis
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Non-invasive Brain-Computer Interfaces (BCI) offer a safe and accessible means of connecting the human brain to external devices, with broad applications in home and clinical settings to enhance human capabilities. However, the high noise level and limited task-specific data in non-invasive signals constrain decoding capabilities. Recently, the adoption of self-supervised pre-training is transforming the landscape of non-invasive BCI research, enabling the development of brain foundation models to capture generic neural representations from large-scale unlabeled electroencephalography (EEG) signals with substantial noises. However, despite these advances, the field currently lacks comprehensive, practical and extensible benchmarks to assess the utility of the public foundation models across diverse BCI tasks, hindering their widespread adoption. To address this challenge, we present AdaBrain-Bench, a large-scale standardized benchmark to systematically evaluate brain foundation models in widespread non-invasive BCI tasks. AdaBrain-Bench encompasses a diverse collection of representative BCI decoding datasets spanning 7 key applications. It introduces a streamlined task adaptation pipeline integrated with multi-dimensional evaluation metrics and a set of adaptation tools. The benchmark delivers an inclusive framework for assessing generalizability of brain foundation models across key transfer settings, including cross-subject, multi-subject, and few-shot scenarios. We leverage AdaBrain-Bench to evaluate a suite of publicly available brain foundation models and offer insights into practices for selecting appropriate models in various scenarios. We make our benchmark pipeline available to enable reproducible research and external use, offering a continuously evolving platform to foster progress toward robust and generalized neural decoding solutions.

Problem

• Lack of benchmarks for brain foundation models in BCI tasks
• High noise and limited data in non-invasive EEG signals
• Need for standardized evaluation of model generalizability across scenarios

Innovation

• Self-supervised pre-training for EEG signals
• Large-scale standardized benchmark AdaBrain-Bench
• Multi-dimensional evaluation metrics pipeline