EmT: A Novel Transformer for Generalized Cross-subject EEG Emotion Recognition

2024-06-26

arXiv.org

AI Summary

Existing cross-subject EEG-based emotion recognition methods struggle to effectively model long-range temporal dependencies associated with affective cognition. To address this, we propose EmT, the first model that represents EEG signals as dynamic temporal graph sequences. EmT integrates three novel components: Temporal Graph Construction (TGC), Residual Multi-View Pyramid Graph Convolutional Networks (RMPG), and a Dual Hybrid Mechanism Temporal Context Transformer (TCT), collectively enabling explicit modeling of long-range cognitive temporal dependencies. The architecture unifies support for both emotion classification and regression tasks, with task-specific output modules ensuring flexible adaptation. Evaluated on four benchmark public EEG datasets, EmT achieves state-of-the-art performance in cross-subject emotion classification and regression, significantly outperforming existing graph neural network and Transformer-based baselines.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Humans and AI: Emotional Intelligence

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Integrating prior knowledge of neurophysiology into neural network architecture enhances the performance of emotion decoding. While numerous techniques emphasize learning spatial and short-term temporal patterns, there has been limited emphasis on capturing the vital long-term contextual information associated with emotional cognitive processes. In order to address this discrepancy, we introduce a novel transformer model called emotion transformer (EmT). EmT is designed to excel in both generalized cross-subject EEG emotion classification and regression tasks. In EmT, EEG signals are transformed into a temporal graph format, creating a sequence of EEG feature graphs using a temporal graph construction module (TGC). A novel residual multi-view pyramid GCN module (RMPG) is then proposed to learn dynamic graph representations for each EEG feature graph within the series, and the learned representations of each graph are fused into one token. Furthermore, we design a temporal contextual transformer module (TCT) with two types of token mixers to learn the temporal contextual information. Finally, the task-specific output module (TSO) generates the desired outputs. Experiments on four publicly available datasets show that EmT achieves higher results than the baseline methods for both EEG emotion classification and regression tasks. The code is available at https://github.com/yi-ding-cs/EmT.

Problem

• Enhances EEG emotion recognition using neurophysiology knowledge.
• Captures long-term contextual information in emotional processes.
• Improves cross-subject EEG emotion classification and regression tasks.

Innovation

• Transformer model integrates neurophysiology for EEG emotion recognition
• Temporal graph construction transforms EEG signals into feature graphs
• Residual multi-view pyramid GCN learns dynamic graph representations