HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Prediction

2026-09-16

AI Summary

研究提出HAP框架，通过观察手部动作和推断目标情境来预测未来头部6-DoF运动，利用动态遮挡图和因果时间推理提高预测准确性。

Abstract

Egocentric motion forecasting has primarily focused on hands and manipulated objects, leaving future human head motion comparatively underexplored. During manipulation, the head both redirects perception toward the target to acquire task-relevant evidence and coordinates with body and hand motion. We therefore formulate future six Degree of Freedom (6-DoF) head-motion prediction conditioned on observed hand motion and inferred target context, and propose HAP, a Hand-Driven Active Perception framework. HAP infers confidence for each target object from observed hand motion and object geometry. Then constructs a dynamic Predictive Target-Centric Amodal Occlusion Graph (P-TAOG) representing current and potential occlusion among candidate objects. Directed graph and causal temporal reasoning encode the evolving target conditioned perceptual state, which is fused with hand and head motion history. A horizon-wise gate then blends the learned trajectory with a constant velocity prior. We further introduce Bottle, an egocentric RGB-D dataset of object manipulation toward specified targets, with coordinated head and hand motion under changing target visibility. Experiments on the public dataset and Bottle show that HAP achieves lower head motion prediction errors than representative baselines, supporting the value of hand driven intention and dynamic occlusion reasoning for anticipating human head motion. Code will be released at https://HAP-ego.github.io/HAP.

Problem

• Egocentric Motion Forecasting
• Head Motion Prediction
• Hand-Driven Active Perception

Innovation

• Hand-Driven Active Perception
• 6-DoF head-motion prediction
• Predictive Target-Centric Amodal Occlusion Graph (P-TAOG)
• Egocentric RGB-D dataset