DynaHOI: Benchmarking Hand-Object Interaction for Dynamic Target

2026-02-12

AI Summary

This work addresses the limitation of existing hand motion generation benchmarks, which predominantly focus on static objects and struggle to evaluate dynamic object interactions and temporally sensitive coordination. To this end, we present DynaHOI-10M, the first large-scale benchmark systematically designed for dynamic hand-object interaction, encompassing eight major categories and 22 fine-grained motion patterns, with 10 million frames and 180K trajectories. We also introduce DynaHOI-Gym, a unified online closed-loop evaluation platform. Building upon this benchmark, we propose ObAct, a baseline method that integrates short-term historical observations with current states through a spatiotemporal attention mechanism and a parametric action generator, evaluated using rollout-based closed-loop metrics. Experiments demonstrate that ObAct improves position success rate by 8.1%, validating the efficacy of explicit dynamic interaction modeling.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Most existing hand motion generation benchmarks for hand-object interaction (HOI) focus on static objects, leaving dynamic scenarios with moving targets and time-critical coordination largely untested. To address this gap, we introduce the DynaHOI-Gym, a unified online closed-loop platform with parameterized motion generators and rollout-based metrics for dynamic capture evaluation. Built on DynaHOI-Gym, we release DynaHOI-10M, a large-scale benchmark with 10M frames and 180K hand capture trajectories, whose target motions are organized into 8 major categories and 22 fine-grained subcategories. We also provide a simple observe-before-act baseline (ObAct) that integrates short-term observations with the current frame via spatiotemporal attention to predict actions, achieving an 8.1% improvement in location success rate.

Problem

• hand-object interaction
• dynamic targets
• motion generation
• benchmarking
• time-critical coordination

Innovation

• hand-object interaction
• dynamic target
• closed-loop benchmark
• spatiotemporal attention
• motion generation