Hand Visibility Detector: Per-Keypoint Visibility Estimation for Hands

2026-08-11

AI Summary

This work addresses the common oversight in existing hand pose estimation methods—namely, the neglect of joint visibility—which hinders reliable assessment of estimation quality under occlusion. The study introduces joint visibility estimation as a standalone task and proposes a visibility detector built upon a large-scale pretrained hand pose model. Furthermore, it integrates a visibility-weighted multi-view triangulation strategy to refine 3D pose reconstruction. The proposed approach substantially improves visibility prediction accuracy and effectively reduces reprojection error in 3D hand pose annotation, thereby demonstrating the practical utility of explicit visibility estimation. To facilitate adoption and further research, the authors release a ready-to-use toolkit alongside their findings.

Abstract

Hand Pose Estimation (HPE) is a fundamental technology for various applications such as AR/VR and robotics. In these applications, the visibility of each hand joint in the image is crucial for assessing the reliability of estimation results under occlusion. However, most existing HPE methods output joint positions without explicitly indicating their visibility. Although some methods account for occlusion or visibility, visibility estimation has mainly been used as an auxiliary signal for improving pose estimation. To our knowledge, per-joint hand visibility estimation has not been systematically studied as a standalone task. In this work, we propose Hand Visibility Detector, a model for estimating the visibility of individual hand joints, and present the first systematic investigation of visibility estimation as an independent task. We show that leveraging the prior knowledge of HPE models pretrained on large-scale data as a backbone yields high performance in this task. We further demonstrate the utility of Hand Visibility Detector on a downstream task of 3D hand pose annotation via multi-view triangulation of 2D keypoints, showing that visibility-weighted triangulation reduces reprojection error. Our method is released as a ready-to-use package, and the code and demo are available at https://github.com/ryhara/hand_visibility_detector .

Problem

• Hand Pose Estimation
• Visibility Est日晚间tion
• Occlusion
• Keypoint Visibility
• 3D Hand Pose

Innovation

• hand visibility estimation
• per-keypoint visibility
• hand pose estimation
• visibility-weighted triangulation
• occlusion handling