Problem
Research questions and friction points this paper is trying to address.
Robot Learning
Human-Robot Interaction
Unsupervised Learning
SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers
🤖 AI Summary
To address the scarcity of real-world human motion data and poor generalization in human-robot handover interactions, this paper proposes the first end-to-end hand-object cooperative motion synthesis framework. Our method jointly optimizes hand pose and object pose via generative modeling, incorporates physics-based constraints to guide physically plausible motion synthesis, and employs a simulation-to-reality transfer strategy. Crucially, it requires no real motion-capture data—training samples covering orders-of-magnitude more objects and grasps are generated solely from synthetic data. In both simulated and real robotic platforms, our approach matches state-of-the-art methods that rely on real human motion data. Moreover, on a newly constructed large-scale benchmark comprising unseen objects and novel handover actions, it significantly improves robustness and scalability, demonstrating superior generalization capability.
📝 Abstract
Vision-based human-to-robot handover is an important and challenging task in human-robot interaction. Recent work has attempted to train robot policies by interacting with dynamic virtual humans in simulated environments, where the policies can later be transferred to the real world. However, a major bottleneck is the reliance on human motion capture data, which is expensive to acquire and difficult to scale to arbitrary objects and human grasping motions. In this paper, we introduce a framework that can generate plausible human grasping motions suitable for training the robot. To achieve this, we propose a hand-object synthesis method that is designed to generate handover-friendly motions similar to humans. This allows us to generate synthetic training and testing data with 100x more objects than previous work. In our experiments, we show that our method trained purely with synthetic data is competitive with state-of-the-art methods that rely on real human motion data both in simulation and on a real system. In addition, we can perform evaluations on a larger scale compared to prior work. With our newly introduced test set, we show that our model can better scale to a large variety of unseen objects and human motions compared to the baselines.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Realistic Hand-Object Interaction Synthesis
Robot Training Efficiency
Simulated Data Effectiveness
🔎 Similar Papers
No similar papers found.
