Physics Informed Human Posture Estimation Based on 3D Landmarks from Monocular RGB-Videos

2025-12-07

AI Summary

To address anatomical implausibility and insufficient robustness in monocular RGB video-based 3D human pose estimation, this paper proposes a real-time, anatomy-aware optimization framework integrating physics-based priors with deep learning. Methodologically, it refines BlazePose’s 2D/3D keypoint outputs by incorporating subject-specific bone-length modeling, biomechanical constraints—including joint angle limits and kinematic connectivity—and a bone-length physical penalty term. An adaptive-confidence Kalman filter dynamically calibrates anatomical parameters without retraining the base model. Evaluated on Physio2.2M, the method reduces 3D MPJPE by 10.2% and joint angle error by 16.6%, while enabling real-time inference on consumer-grade edge devices with on-device privacy preservation. The core contribution is a lightweight, anatomy-consistency-driven optimization paradigm that significantly enhances reliability and practicality for clinical applications—such as physical therapy and sports coaching—without compromising computational efficiency.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: State Estimation
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Applications providing automated coaching for physical training are increasing in popularity, for example physical therapy. These applications rely on accurate and robust pose estimation using monocular video streams. State-of-the-art models like BlazePose excel in real-time pose tracking, but their lack of anatomical constraints indicates improvement potential by including physical knowledge. We present a real-time post-processing algorithm fusing the strengths of BlazePose 3D and 2D estimations using a weighted optimization, penalizing deviations from expected bone length and biomechanical models. Bone length estimations are refined to the individual anatomy using a Kalman filter with adapting measurement trust. Evaluation using the Physio2.2M dataset shows a 10.2 percent reduction in 3D MPJPE and a 16.6 percent decrease in errors of angles between body segments compared to BlazePose 3D estimation. Our method provides a robust, anatomically consistent pose estimation based on a computationally efficient video-to-3D pose estimation, suitable for automated physiotherapy, healthcare, and sports coaching on consumer-level laptops and mobile devices. The refinement runs on the backend with anonymized data only.

Problem

• Enhance 3D human pose estimation from monocular RGB videos
• Incorporate anatomical constraints to improve biomechanical accuracy
• Enable real-time, robust pose tracking for automated coaching applications

Innovation

• Real-time post-processing algorithm with weighted optimization
• Kalman filter refines bone length using anatomical constraints
• Reduces 3D pose errors by penalizing biomechanical deviations