Generating Robot Hands from Human Demonstrations

2026-06-18

AI Summary

This work addresses the challenges of combinatorial explosion and morphological learning difficulty in co-optimizing robot embodiment and control by proposing a data-driven generative framework. Leveraging 4 million frames of human fingertip motion data, the method employs an inverse kinematics–matched minimal control policy to directly generate manufacturable, tree-structured robotic hands. The approach integrates reinforcement learning to accelerate design search, topology optimization, and a bioinspired, monolithic-printable joint mechanism, enabling end-to-end generation. The resulting six-degree-of-freedom general-purpose hand and three-degree-of-freedom task-specific hand demonstrate superior fingertip tracking accuracy compared to commercial counterparts in real-world experiments, while substantially reducing mechanical complexity.

Abstract

Robot learning has advanced rapidly in learning control, but learning the physical body of a robot remains much more difficult because jointly searching over design and control creates a very large combinatorial problem. Here, we present a data-driven framework for generating robot hands from human demonstrations. Instead of learning a complex controller together with each candidate design, we generate robot hand designs using the same simple control policy used after fabrication: matching fingertip positions through inverse kinematics. Using more than 4 million frames of human fingertip motion from everyday manipulation, our algorithm optimizes tree-structured robot hands to reproduce desired target motions. The framework produced both a 6-degree-of-freedom (DoF) general-purpose hand and lower-DoF task-specific hands with spatial four-bar mimic joints. To accelerate the search over designs, we trained a reinforcement-learning (RL) actor to propose good hand designs and joint angles, reducing search time from hours to minutes. We fabricated the mechanisms directly as one-piece articulated structures with print-in-place joints. In real-world experiments, the 6-DoF hand achieved highly accurate teleoperated fingertip tracking better than available commercial robot hands, whereas the specialized 3-DoF hands reproduced structured human and synthetic trajectories with reduced mechanical complexity. These results showed that large-scale human motion data can be used not only to train robot controllers but also as a reference for optimizing and generating the physical embodiment of robots.

Problem

• robot hand design
• human demonstration
• embodiment optimization
• inverse kinematics
• combinatorial design

Innovation

• data-driven robot design
• inverse kinematics control
• human demonstration
• reinforcement learning for design
• print-in-place joints