NeuroGaze-Distill: Brain-informed Distillation and Depression-Inspired Geometric Priors for Robust Facial Emotion Recognition

2025-09-15

AI Summary

Existing pixel-only facial emotion recognition (FER) models suffer from weak cross-dataset generalization due to the indirectness and bias inherent in facial expressions as emotional proxies. To address this, we propose NeuroGaze-Distill: a cross-modal knowledge distillation framework that transfers EEG-informed static valence/arousal (V/A) prototype grids—guided by depression research—and geometric priors over embedding space—without requiring paired multimodal data—to ResNet-18/50-based image models. Our method integrates Prototype-based Knowledge Distillation (Proto-KD), enforcing cosine similarity alignment between student and teacher prototypes, with D-Geo regularization to preserve discriminative embedding geometry. It jointly optimizes classification cross-entropy and distillation losses. Trained on FERPlus, NeuroGaze-Distill achieves consistent improvements across FERPlus, AffectNet-mini, and other benchmarks, with particularly strong gains in cross-domain settings. A compact 5×5 prototype grid suffices for optimal performance, demonstrating both efficacy and architectural simplicity.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Facial emotion recognition (FER) models trained only on pixels often fail to generalize across datasets because facial appearance is an indirect and biased proxy for underlying affect. We present NeuroGaze-Distill, a cross-modal distillation framework that transfers brain-informed priors into an image-only FER student via static Valence/Arousal (V/A) prototypes and a depression-inspired geometric prior (D-Geo). A teacher trained on EEG topographic maps from DREAMER (with MAHNOB-HCI as unlabeled support) produces a consolidated 5x5 V/A prototype grid that is frozen and reused; no EEG-face pairing and no non-visual signals at deployment are required. The student (ResNet-18/50) is trained on FERPlus with conventional CE/KD and two lightweight regularizers: (i) Proto-KD (cosine) aligns student features to the static prototypes; (ii) D-Geo softly shapes the embedding geometry in line with affective findings often reported in depression research (e.g., anhedonia-like contraction in high-valence regions). We evaluate both within-domain (FERPlus validation) and cross-dataset protocols (AffectNet-mini; optional CK+), reporting standard 8-way scores alongside present-only Macro-F1 and balanced accuracy to fairly handle label-set mismatch. Ablations attribute consistent gains to prototypes and D-Geo, and favor 5x5 over denser grids for stability. The method is simple, deployable, and improves robustness without architectural complexity.

Problem

• Improving facial emotion recognition generalization across datasets
• Transferring brain-informed priors into image-only models
• Enhancing robustness with depression-inspired geometric constraints

Innovation

• Cross-modal distillation from EEG to image models
• Static valence/arousal prototypes for feature alignment
• Depression-inspired geometric priors for embedding regularization