Benchmarking 3D Human Pose Estimation Models Under Occlusions

2025-04-14

AI Summary

This work addresses the limited robustness and generalization of 3D human pose estimation (HPE) models under occlusion, varying camera poses, and diverse human motions. To this end, we introduce BlendMimic3D—a synthetically generated benchmark dataset that unifies 2D detection inputs and 3D keypoint representations—and establish a systematic evaluation protocol covering multi-view setups, multiple occlusion types (rigid, dynamic, and semantic), and motion variability. We quantitatively demonstrate, for the first time, that state-of-the-art models exhibit high sensitivity to occlusion (average error increases by 47% under moderate occlusion) and camera pose variations. We propose a cross-dataset keypoint format alignment paradigm to enable realistic generalization assessment. Our methodology integrates Blender-based rendering with MimicMotion-driven animation, multi-camera simulation, and a rigorous robustness quantification framework. BlendMimic3D provides a reproducible benchmark and a structured pathway for advancing robust 3D HPE.

Technology Category

• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper addresses critical challenges in 3D Human Pose Estimation (HPE) by analyzing the robustness and sensitivity of existing models to occlusions, camera position, and action variability. Using a novel synthetic dataset, BlendMimic3D, which includes diverse scenarios with multi-camera setups and several occlusion types, we conduct specific tests on several state-of-the-art models. Our study focuses on the discrepancy in keypoint formats between common datasets such as Human3.6M, and 2D datasets such as COCO, commonly used for 2D detection models and frequently input of 3D HPE models. Our work explores the impact of occlusions on model performance and the generality of models trained exclusively under standard conditions. The findings suggest significant sensitivity to occlusions and camera settings, revealing a need for models that better adapt to real-world variability and occlusion scenarios. This research contributed to ongoing efforts to improve the fidelity and applicability of 3D HPE systems in complex environments.

Problem

• Assessing 3D pose model robustness to occlusions and camera variations
• Analyzing keypoint format discrepancies between 2D and 3D datasets
• Evaluating model performance gaps in real-world occlusion scenarios

Innovation

• Uses BlendMimic3D synthetic dataset
• Tests models on occlusion robustness
• Analyzes keypoint format discrepancies