A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

2026-04-29

AI Summary

This work addresses the limitations of existing unimodal epileptic seizure detection methods, which are prone to interference from motion artifacts or benign behaviors and rely on time-consuming manual review of synchronized video-EEG data. The authors propose EEGVFusion, a novel framework that integrates self-supervised EEG representation learning, 3D convolutional video encoding, optimal transport-based alignment, and bidirectional cross-attention mechanisms to enable synergistic multimodal analysis of neural and behavioral signals. Evaluated on an expert-annotated mouse EEG-video dataset, the method achieves a balanced accuracy of 0.9957 and a false-positive rate of 0.6250 FP/h under random split evaluation. In the more challenging leave-one-subject-out setting, it further reduces the false-positive rate to 0.4833 FP/h while maintaining 100% event sensitivity, substantially outperforming current state-of-the-art approaches.

Abstract

Reliable seizure detection in mouse models is essential for preclinical epilepsy research, yet manual review of synchronized video-EEG recordings is labor-intensive and single-modality systems fail for complementary reasons: video-based methods are easily confounded by benign behaviors, whereas EEG-based methods are vulnerable to ictal motion artifacts. We present EEGVFusion, a multimodal framework that combines self-supervised EEG representation learning, spatio-temporal video encoding, optimal-transport alignment, and bidirectional cross-attention to integrate neural and behavioral evidence. We also curate an expert-annotated dataset of synchronized EEG and video recordings comprising 93 sessions from 15 mice for training and evaluation. In the random-session split, EEGVFusion achieved a Balanced Accuracy of 0.9957 with perfect event sensitivity and an Event FAR of 0.6250 FP/h, indicating strong seizure detection performance with a low false-alarm burden. In a single held-out-subject evaluation with Subject 110 reserved for testing, EEGVFusion achieved a Balanced Accuracy of 0.9718 and reduced Event FAR from 2.7250 FP/h for the EEG-only counterpart to 0.4833 FP/h while preserving perfect event sensitivity. Targeted ablations further showed that EEG pre-training and OT alignment help reduce false alarms while preserving event sensitivity.

Problem

• seizure detection
• multimodal integration
• EEG-video synchronization
• motion artifacts
• false alarms

Innovation

• multimodal fusion
• self-supervised EEG representation
• optimal transport alignment
• cross-attention
• seizure detection