DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation

2025-09-04

AI Summary

Addressing the challenges of difficult dexterous manipulation data acquisition and inefficient skill transfer to robots, this paper proposes the “perioperation” paradigm. We design DEXOP, a passive hand exoskeleton, featuring a novel mechanical coupling mechanism between human and robotic fingers to enable real-time pose mirroring and proprioceptive force feedback—significantly enhancing naturalness of human manipulation and fidelity of collected data. The system integrates vision and tactile sensing with passive mechanical linkage, enabling high-precision, multimodal, contact-intensive dexterous manipulation data collection. Experiments on diverse high-difficulty contact tasks demonstrate substantially higher per-unit-time data utilization compared to conventional teleoperation, alongside markedly improved policy generalizability and task success rates. Our core contribution is the first dexterous manipulation data acquisition and transfer framework that simultaneously achieves high-fidelity perception, low-intrusion human–robot interaction, and efficient skill migration.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Interaction Techniques and Devices
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We introduce perioperation, a paradigm for robotic data collection that sensorizes and records human manipulation while maximizing the transferability of the data to real robots. We implement this paradigm in DEXOP, a passive hand exoskeleton designed to maximize human ability to collect rich sensory (vision + tactile) data for diverse dexterous manipulation tasks in natural environments. DEXOP mechanically connects human fingers to robot fingers, providing users with direct contact feedback (via proprioception) and mirrors the human hand pose to the passive robot hand to maximize the transfer of demonstrated skills to the robot. The force feedback and pose mirroring make task demonstrations more natural for humans compared to teleoperation, increasing both speed and accuracy. We evaluate DEXOP across a range of dexterous, contact-rich tasks, demonstrating its ability to collect high-quality demonstration data at scale. Policies learned with DEXOP data significantly improve task performance per unit time of data collection compared to teleoperation, making DEXOP a powerful tool for advancing robot dexterity. Our project page is at https://dex-op.github.io.

Problem

• Transfer human manipulation skills to robots
• Collect rich sensory data for dexterous tasks
• Improve robot learning efficiency over teleoperation

Innovation

• Passive hand exoskeleton for data collection
• Mechanical finger connection with force feedback
• Pose mirroring from human to robot hand