STROKEVISION-BENCH: A Multimodal Video And 2D Pose Benchmark For Tracking Stroke Recovery

2025-09-02

AI Summary

Clinical assessment of upper-limb function in stroke survivors has long relied on subjective, non-quantitative scoring scales, with no dedicated multimodal dataset aligned to standardized clinical tasks—such as the Box and Block Test (BBT)—to support objective motion quantification. Method: We introduce StrokeVision-Bench, the first multimodal benchmark dataset specifically designed for stroke rehabilitation assessment, comprising 1,000 BBT videos captured from stroke patients, each annotated with 2D skeletal keypoints. We further propose a video-skeleton fusion framework for action recognition and establish a unified evaluation benchmark. Contribution/Results: StrokeVision-Bench bridges a critical data gap in quantitative motor assessment, significantly enhancing objectivity, reproducibility, and scalability of upper-limb functional evaluation. It serves as foundational infrastructure and methodological support for automated, data-driven stroke rehabilitation assessment.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Despite advancements in rehabilitation protocols, clinical assessment of upper extremity (UE) function after stroke largely remains subjective, relying heavily on therapist observation and coarse scoring systems. This subjectivity limits the sensitivity of assessments to detect subtle motor improvements, which are critical for personalized rehabilitation planning. Recent progress in computer vision offers promising avenues for enabling objective, quantitative, and scalable assessment of UE motor function. Among standardized tests, the Box and Block Test (BBT) is widely utilized for measuring gross manual dexterity and tracking stroke recovery, providing a structured setting that lends itself well to computational analysis. However, existing datasets targeting stroke rehabilitation primarily focus on daily living activities and often fail to capture clinically structured assessments such as block transfer tasks. Furthermore, many available datasets include a mixture of healthy and stroke-affected individuals, limiting their specificity and clinical utility. To address these critical gaps, we introduce StrokeVision-Bench, the first-ever dedicated dataset of stroke patients performing clinically structured block transfer tasks. StrokeVision-Bench comprises 1,000 annotated videos categorized into four clinically meaningful action classes, with each sample represented in two modalities: raw video frames and 2D skeletal keypoints. We benchmark several state-of-the-art video action recognition and skeleton-based action classification methods to establish performance baselines for this domain and facilitate future research in automated stroke rehabilitation assessment.

Problem

• Objective assessment of stroke upper extremity recovery
• Addressing gaps in clinically structured rehabilitation datasets
• Benchmarking automated methods for stroke rehabilitation tracking

Innovation

• Introduces StrokeVision-Bench dataset for stroke patients
• Uses multimodal video and 2D skeletal keypoints data
• Benchmarks action recognition methods for rehabilitation assessment