See the Emotion: A Facial Emoji Proxy Modeling for EEG Emotion Recognition

2026-07-02

AI Summary

This study addresses the limited interpretability of existing EEG-based emotion recognition models, which struggle to establish semantic links between neural activity and human-understandable emotional states. To bridge this gap, the authors propose a cross-modal generative framework that maps high-dimensional EEG signals onto a dynamic manifold of facial-expression emojis, enabling both visualizable and anonymized emotion representation. The approach employs FMENet as the backbone to model spatial coherence and introduces a Facial Emoji Learning Branch (FELB) as a structured semantic regularizer to facilitate cross-modal reconstruction from EEG to expressive emojis. Evaluated on the EAV and MMER benchmarks, the model achieves state-of-the-art accuracy in unimodal EEG-based emotion recognition while generating semantically faithful facial animations that intuitively reveal the temporal evolution of brain-emotion dynamics.

Abstract

Despite the high accuracy of EEG-based emotion recognition, existing models remain opaque "black boxes", lacking semantic grounding between abstract neural features and human-interpretable states. In this paper, we reframe EEG explainability as a cross-modal generation task, shifting the paradigm from feature attribution to behavioral visualization. We introduce Facial Emoji Proxy Modeling, a novel framework that translates high-dimensional EEG signals into identity-anonymized facial emojis. Guided by the neuroscientific inspiration of neural-facial association, this approach grounds neural representations in the manifold of observable facial dynamics. Technically, our framework integrates FMENet, a specialized backbone modeling expression-relevant spatial synergies, and the Facial Emoji Learning Branch (FELB), which treats emoji reconstruction as a structured semantic regularizer. Extensive experiments on EAV and MMER benchmarks demonstrate that our method achieves state-of-the-art accuracy among EEG-only models. Crucially, it generates semantically faithful facial animations that provide a transparent, privacy-preserving window into the brain's emotional evolution, effectively allowing users to "see the emotion" directly from neural signals. Code is available at https://github.com/xian-sh/SeeEmotion

Problem

• EEG emotion recognition
• model interpretability
• neural-facial association
• cross-modal generation
• semantic grounding

Innovation

• Facial Emoji Proxy Modeling
• EEG Emotion Recognition
• Cross-modal Generation
• Explainable AI
• Neural-Facial Association