STAMP: Spatial-Temporal Adapter with Multi-Head Pooling

2025-11-13

AI Summary

Prior work lacks a systematic comparison of EEG-specific foundation models (EEGFMs) versus general-purpose time-series foundation models (TSFMs) on EEG analysis tasks. Method: We propose STAMP, a lightweight spatial-temporal adapter that integrates EEG’s inter-channel spatial dependencies and temporal dynamics implicitly—without modifying pretrained TSFM parameters—via univariate embedding and multi-head pooling. Contribution/Results: This is the first comprehensive evaluation of TSFMs’ applicability to EEG analysis, demonstrating their competitiveness with dedicated EEGFMs. STAMP achieves state-of-the-art performance across eight clinical EEG classification benchmarks, with significantly reduced parameter count and training cost. It enables efficient, parameter-efficient transfer of general-purpose TSFMs to EEG domains, advancing scalable and adaptable neurophysiological signal modeling.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Time series foundation models (TSFMs) pretrained on data from multiple domains have shown strong performance on diverse modeling tasks. Various efforts have been made to develop foundation models specific to electroencephalography (EEG) data, which records brain electrical activity as time series. However, no comparative analysis of EEG-specific foundation models (EEGFMs) versus general TSFMs has been performed on EEG-specific tasks. We introduce a novel Spatial-Temporal Adapter with Multi-Head Pooling (STAMP), which leverages univariate embeddings produced by a general TSFM, implicitly models spatial-temporal characteristics of EEG data, and achieves performance comparable to state-of-the-art EEGFMs. A comprehensive analysis is performed on 8 benchmark datasets of clinical tasks using EEG for classification, along with ablation studies. Our proposed adapter is lightweight in trainable parameters and flexible in the inputs it can accommodate, supporting easy modeling of EEG data using TSFMs.

Problem

• Compares EEG-specific versus general time series foundation models
• Introduces lightweight adapter for modeling EEG spatial-temporal characteristics
• Evaluates performance on clinical EEG classification tasks across datasets

Innovation

• Spatial-Temporal Adapter with Multi-Head Pooling
• Leverages univariate embeddings from general TSFM
• Lightweight adapter flexibly models EEG data characteristics