EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

2026-04-14

AI Summary

Accurately assessing psychological states requires effective integration of electroencephalography (EEG) and multimodal signals, with the core challenge lying in modeling the hierarchical structures inherent in heterogeneous modalities. To address this, this work proposes the EEG-MoCE framework, which introduces—for the first time—a hyperbolic mixture-of-experts mechanism with learnable curvature. Each modality is embedded into hyperbolic space via dedicated expert modules, where the curvature is adaptively adjusted to align with its intrinsic geometric properties. Furthermore, a curvature-aware dynamic fusion strategy is devised to enable hierarchy-sensitive information integration. The proposed method significantly outperforms existing approaches across multiple tasks—including emotion recognition, sleep staging, and cognitive assessment—achieving state-of-the-art performance on several benchmark datasets.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Electroencephalography (EEG)-based multimodal learning integrates brain signals with complementary modalities to improve mental state assessment, providing great clinical potential. The effectiveness of such paradigms largely depends on the representation learning on heterogeneous modalities. For EEG-based paradigms, one promising approach is to leverage their hierarchical structures, as recent studies have shown that both EEG and associated modalities (e.g., facial expressions) exhibit hierarchical structures reflecting complex cognitive processes. However, Euclidean embeddings struggle to represent these hierarchical structures due to their flat geometry, while hyperbolic spaces, with their exponential growth property, are naturally suited for them. In this work, we propose EEG-MoCE, a novel hyperbolic mixture-of-curvature experts framework designed for multimodal neurotechnology. EEG-MoCE assigns each modality to an expert in a learnable-curvature hyperbolic space, enabling adaptive modeling of its intrinsic geometry. A curvature-aware fusion strategy then dynamically weights experts, emphasizing modalities with richer hierarchical information. Extensive experiments on benchmark datasets demonstrate that EEG-MoCE achieves state-of-the-art performance, including emotion recognition, sleep staging, and cognitive assessment.

Problem

• EEG
• multimodal learning
• hierarchical structure
• representation learning
• mental state assessment

Innovation

• hyperbolic geometry
• mixture-of-experts
• EEG-based multimodal learning
• learnable curvature
• hierarchical representation