Leveraging Anthropometric Measurements to Improve Human Mesh Estimation and Ensure Consistent Body Shapes

2024-09-26

arXiv.org

AI Summary

Existing single-video human mesh estimation (HME) methods suffer from inter-frame inconsistency in the canonical T-pose and suboptimal body shape parameter accuracy compared to high-precision 3D human pose estimation (HPE). This work proposes an HME optimization framework integrating anthropometric priors with accurate HPE. Specifically, it introduces the first differentiable, direct mapping from measurable anthropometric dimensions—such as chest and waist circumferences—to SMPL shape parameters. An A2B neural network is designed for robust shape parameter regression, while inverse kinematics (IK) ensures temporal consistency and geometric plausibility across frames. Evaluated on ASPset and fit3D, the method reduces MPJPE by over 30 mm. When replacing the shape module of existing HME pipelines, it guarantees strict intra-video shape consistency while significantly improving mesh reconstruction accuracy.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Video Understanding & Activity Analysis
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The basic body shape (i.e., the body shape in T-pose) of a person does not change within a single video. However, most SOTA human mesh estimation (HME) models output a slightly different, thus inconsistent basic body shape for each video frame. Furthermore, we find that SOTA 3D human pose estimation (HPE) models outperform HME models regarding the precision of the estimated 3D keypoint positions. We solve the problem of inconsistent body shapes by leveraging anthropometric measurements like taken by tailors from humans. We create a model called A2B that converts given anthropometric measurements to basic body shape parameters of human mesh models. We obtain superior and consistent human meshes by combining the A2B model results with the keypoints of 3D HPE models using inverse kinematics. We evaluate our approach on challenging datasets like ASPset or fit3D, where we can lower the MPJPE by over 30 mm compared to SOTA HME models. Further, replacing estimates of the body shape parameters from existing HME models with A2B results not only increases the performance of these HME models, but also guarantees consistent body shapes.

Problem

• Inconsistent basic body shape output in video frames by HME models
• Leveraging anthropometric measurements to ensure consistent body shapes
• Improving human mesh estimation using A2B model and 3D HPE keypoints

Innovation

• Uses anthropometric measurements for body shape
• Converts measurements to shape parameters via A2B
• Combines A2B with HPE using inverse kinematics