CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

2026-07-10

AI Summary

This work addresses the limitations of existing mask-based reconstruction approaches for EEG foundation models, which suffer from degraded performance under high noise levels and narrow bandwidth signals. To overcome these challenges, the authors propose a hybrid architecture that integrates multi-scale temporal convolutions with a Transformer encoder and, for the first time, incorporates contrastive learning into self-supervised pretraining for EEG. This design better captures the time-frequency characteristics inherent in EEG signals, substantially improving data efficiency and model generalization. Experimental results demonstrate that the proposed model achieves state-of-the-art or comparable performance across multiple benchmark tasks under various electrode configurations. Notably, even when trained from scratch, it outperforms prior single-task models, validating the effectiveness of both the architectural design and the contrastive pretraining strategy.

Abstract

Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.

Problem

• EEG decoding
• self-supervised pretraining
• contrastive learning
• foundation models
• noisy EEG data

Innovation

• contrastive pretraining
• multiscale convolution
• Transformer
• EEG decoding
• foundation model