Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology

2026-05-05

AI Summary

This study presents the first validation of zero-shot recognition capabilities of general-purpose multimodal large language models (MLLMs) for epileptic semiology analysis, demonstrating their ability to automatically interpret 20 ILAE-defined pathological motor features in clinical videos without task-specific fine-tuning. The approach integrates preprocessing strategies including facial cropping, pose estimation, and audio denoising, and is evaluated against CNN and ViT baselines on a dataset of 90 clinical video segments. Results show that MLLMs outperform fine-tuned baselines on 13 out of 18 assessed features, with 94.3% of correct predictions accompanied by highly faithful (≥60%) interpretable reasoning. This work establishes a novel paradigm for epilepsy video analysis that is both highly interpretable and eliminates the need for model fine-tuning, significantly enhancing diagnostic assistance efficiency.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated robust capabilities in recognizing everyday human activities, yet their potential for analyzing clinically significant involuntary movements in neurological disorders remains largely unexplored. This pilot study evaluates the capability of MLLMs for automated recognition of pathological movements in seizure videos. We assessed the zero-shot performance of state-of-the-art MLLMs on 20 ILAE-defined semiological features across 90 clinical seizure recordings. MLLMs outperformed fine-tuned Convolutional Neural Network (CNN) and Vision Transformer (ViT) baseline models on 13 of 18 features without task-specific training, demonstrating particular strength in recognizing salient postural and contextual features while struggling with subtle, high-frequency movements. Feature-targeted signal enhancement (facial cropping, pose estimation, audio denoising) improved performance on 10 of 20 features. Expert evaluation showed that 94.3 percent of MLLM-generated explanations for correctly predicted cases achieved at least 60 percent faithfulness scores, aligning with epileptologist reasoning. These findings demonstrate the potential of adapting general-purpose MLLMs for specialized clinical video analysis through targeted preprocessing strategies, offering a path toward interpretable, efficient diagnostic assistance. Our code is publicly available at https://github.com/LinaZhangUCLA/PathMotionMLLM.

Problem

• Multimodal Large Language Models
• Pathologic Movements
• Seizure Semiology
• Clinical Video Analysis
• Neurological Disorders

Innovation

• Multimodal Large Language Models
• Seizure Semiology
• Zero-shot Learning
• Pathological Movement Recognition
• Clinical Video Analysis