🤖 AI Summary
This study addresses the performance degradation in egocentric human pose estimation caused by occluded or invisible keypoints, a challenge commonly overlooked by existing methods that neglect visibility disparities. To tackle this issue, the authors introduce keypoint visibility annotations for the first time, construct a large-scale dataset named Eva-3M, and augment the existing EMHI dataset with visibility labels. They further propose EvaPose, the first pose estimation method that explicitly models keypoint visibility. Extensive experiments demonstrate that EvaPose achieves state-of-the-art performance on both Eva-3M and EMHI, confirming that incorporating visibility information significantly enhances the accuracy of visible keypoint prediction.
📝 Abstract
Egocentric human pose estimation (HPE) using a head-mounted device is crucial for various VR and AR applications, but it faces significant challenges due to keypoint invisibility. Nevertheless, none of the existing egocentric HPE datasets provide keypoint visibility annotations, and the existing methods often overlook the invisibility problem, treating visible and invisible keypoints indiscriminately during estimation. As a result, their capacity to accurately predict visible keypoints is compromised. In this paper, we first present Eva-3M, a large-scale egocentric visibility-aware HPE dataset comprising over 3.0M frames, with 435K of them annotated with keypoint visibility labels. Additionally, we augment the existing EMHI dataset with keypoint visibility annotations to further facilitate the research in this direction. Furthermore, we propose EvaPose, a novel egocentric visibility-aware HPE method that explicitly incorporates visibility information to enhance pose estimation accuracy. Extensive experiments validate the significant value of ground-truth visibility labels in egocentric HPE settings, and demonstrate that our EvaPose achieves state-of-the-art performance in both Eva-3M and EMHI datasets.