HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing

2026-08-12

AI Summary

This work addresses the challenge that significant discrepancies in appearance, structure, and viewpoint between human and robotic hands hinder the effective use of large-scale human first-person videos for learning dexterous manipulation in embodied intelligence. To bridge this gap, the paper introduces the first unified image-editing benchmark for human-to-robot hand transfer. By integrating five source datasets and leveraging URDF-conditioned generation alongside large-scale image editing techniques, the authors construct HandEdit—a dataset encompassing 26 robotic hand models and over 200 million edited instances—accompanied by a multidimensional evaluation protocol. Using this benchmark, they systematically evaluate 11 state-of-the-art image editing models, establishing a foundational resource and standardized assessment framework for generalizable embodied robotic learning.

Abstract

Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet its progress is stifled by the high cost of collecting embodiment-aware teleoperation data. While abundant egocentric videos of human hands offer a scalable alternative, the profound discrepancies in appearance, articulation, and camera viewpoints between human and robotic data raise significant challenges for co-training. Though existing general image-editing models demonstrate strong capabilities, they lack necessary embodiment-specific priors to fully bridge this gap. In this work, we present HandEdit, a unified large-scale embodiment-aware image-editing dataset and benchmark specifically designed to transform human hands and arms into various dexterous robotic embodiments within egocentric frames. HandEdit comprises over 200M editing instances derived from five diverse source datasets, covering 26 distinct URDFs, including 13 hand-only and 13 hand-arm configurations. Alongside the dataset, we establish a unified benchmark protocol with two tracks: Hand-only and Hand-Arm, supporting URDF-conditioned evaluation. We conduct extensive evaluations of 11 representative image-editing baselines using a multi-dimensional metric suite, including generic similarity metrics, VLM-based judgment, and embodiment-aware metrics. HandEdit serves as a critical resource at the intersection of image editing and robotics: it advances embodiment-aware editing models while enabling scalable dexterous robotic learning from abundant human video data, paving the way for more generalizable Embodied AI.

Problem

• egocentric vision
• dexterous hand
• human-to-robot transfer
• embodiment-aware editing
• image editing

Innovation

• embodiment-aware editing
• egocentric hand-to-robot translation
• dexterous robotic hands
• URDF-conditioned image editing
• scalable teleoperation