Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach

2025-12-18

AI Summary

This work addresses gaze-primed reaching motion generation—modeling the natural human behavior of first fixating on a target before executing a reaching-and-grasping action. We introduce GazeReach, the first large-scale (23.7K samples) dataset of gaze-primed reaching motions, and propose a diffusion model jointly conditioned on textual descriptions and target pose. We establish the first systematic computational framework for modeling gaze-reaching coordination, design a novel evaluation metric—“Prime Success”—to quantify the plausibility of gaze priming, and construct a cross-dataset benchmark for large-scale gaze-primed motion generation. On the HD-EPIC benchmark, our method achieves 60% Prime Success and 89% Reach Success, significantly outperforming all baselines. These results empirically validate the effectiveness and naturalness of gaze-guided motion generation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Human motion generation is a challenging task that aims to create realistic motion imitating natural human behaviour. We focus on the well-studied behaviour of priming an object/location for pick up or put down -- that is, the spotting of an object/location from a distance, known as gaze priming, followed by the motion of approaching and reaching the target location. To that end, we curate, for the first time, 23.7K gaze-primed human motion sequences for reaching target object locations from five publicly available datasets, i.e., HD-EPIC, MoGaze, HOT3D, ADT, and GIMO. We pre-train a text-conditioned diffusion-based motion generation model, then fine-tune it conditioned on goal pose or location, on our curated sequences. Importantly, we evaluate the ability of the generated motion to imitate natural human movement through several metrics, including the 'Reach Success' and a newly introduced 'Prime Success' metric. On the largest dataset, HD-EPIC, our model achieves 60% prime success and 89% reach success when conditioned on the goal object location.

Problem

• Generates gaze-primed human reaching motion sequences
• Synthesizes realistic body motion for object approach and reach
• Evaluates motion imitation using Prime and Reach Success metrics

Innovation

• Diffusion-based model fine-tuned for gaze-primed reaching
• First curated dataset of 23.7K gaze-primed motion sequences
• Evaluated using novel Prime Success and Reach Success metrics