RePose: A Real-Time 3D Human Pose Estimation and Biomechanical Analysis Framework for Rehabilitation

2026-01-02

arXiv.org

AI Summary

This study addresses the lack of real-time monitoring and precise feedback in current rehabilitation training protocols. To this end, we propose an end-to-end framework for real-time 3D human pose estimation and motion analysis, leveraging multi-view RGB video inputs to enable high-accuracy, low-latency movement tracking and biomechanical assessment. The core innovations include a fast tracking algorithm robust to multi-person interference with sub-millisecond per-frame processing, an enhanced SmoothNet architecture that improves both pose smoothness and accuracy, and a muscle stress visualization module. Integrated into the Unity platform, the system effectively supports patients in performing rehabilitation exercises with proper form, thereby facilitating motor function recovery.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We propose a real-time 3D human pose estimation and motion analysis method termed RePose for rehabilitation training. It is capable of real-time monitoring and evaluation of patients'motion during rehabilitation, providing immediate feedback and guidance to assist patients in executing rehabilitation exercises correctly. Firstly, we introduce a unified pipeline for end-to-end real-time human pose estimation and motion analysis using RGB video input from multiple cameras which can be applied to the field of rehabilitation training. The pipeline can help to monitor and correct patients'actions, thus aiding them in regaining muscle strength and motor functions. Secondly, we propose a fast tracking method for medical rehabilitation scenarios with multiple-person interference, which requires less than 1ms for tracking for a single frame. Additionally, we modify SmoothNet for real-time posture estimation, effectively reducing pose estimation errors and restoring the patient's true motion state, making it visually smoother. Finally, we use Unity platform for real-time monitoring and evaluation of patients'motion during rehabilitation, and to display the muscle stress conditions to assist patients with their rehabilitation training.

Problem

• 3D human pose estimation
• real-time monitoring
• rehabilitation training
• motion analysis
• biomechanical analysis

Innovation

• real-time 3D pose estimation
• rehabilitation framework
• multi-camera RGB input
• fast motion tracking
• SmoothNet refinement