EgoInertia-MI: A Multimodal Egocentric Vision and IMU Benchmark for Motor Impairment Assessment

2026-07-04

AI Summary

Current clinical assessments struggle to continuously capture subtle temporal dynamics of motor impairments, and third-person video recordings face privacy concerns and environmental constraints. To address these limitations, this work presents the first multimodal dataset combining synchronized first-person visual and inertial measurement unit (IMU) data, encompassing 19 upper- and lower-limb activities across three simulated levels of impairment severity, enabling ecologically valid evaluation under natural and privacy-preserving conditions. The study establishes two benchmark tasks—action recognition and impairment severity estimation—and introduces a multimodal fusion approach that achieves Macro-F1 scores of 0.93 and 0.78, respectively, demonstrating the efficacy of the proposed paradigm. This work provides the first first-person multimodal benchmark dataset and evaluation framework for motor impairment assessment.

Abstract

Motor impairments, including tremor, bradykinesia, gait abnormalities, and postural instability, are common across many neurological and movement-related conditions. Conventional clinical assessments are often intermittent and may fail to capture subtle temporal variations in motor behavior. While wearable IMUs and third-person video have shown promise for objective motor assessment, third-person recordings raise privacy concerns and require constrained acquisition setups. In contrast, egocentric vision provides a more naturalistic and privacyaware alternative. In this work, we introduce EgoInertia-MI, a multimodal benchmark dataset combining synchronized egocentric video and wearable IMU signals for motor impairment analysis. The dataset contains 19 upper- and lower-body activities performed by healthy volunteers simulating varying levels of motor impairment severity levels: no impairment, mild impairment, and severe impairment. We establish two benchmark tasks: action recognition and motor impairment severity estimation, and evaluate multiple unimodal and multimodal baselines. Experimental results show that egocentric video provides strong cues for motor impairment assessment, while multimodal fusion achieves the best overall performance, reaching 0.78 Macro-F1 for severity estimation and 0.93 Macro-F1 for action recognition. These findings highlight the potential of combining egocentric vision and wearable sensing for ecologically valid and privacy-aware motor assessment. Code and data are available at:https://fatemah-alh.github.io/EgoInertia-MI-Page/.

Problem

• motor impairment
• egocentric vision
• IMU
• privacy-aware assessment
• neurological disorders

Innovation

• egocentric vision
• IMU
• motor impairment assessment
• multimodal fusion
• benchmark dataset