DEX-Mouse: A Low-cost Portable and Universal Interface with Force Feedback for Data Collection of Dexterous Robotic Hands

2026-04-16

AI Summary

Existing dexterous hand teleoperation systems are commonly hindered by complex calibration procedures, poor portability, limited cross-platform compatibility, and a lack of force feedback, all of which impede the efficient collection of high-quality demonstration data. This work proposes DEX-Mouse—a low-cost, calibration-free, wearable teleoperation interface that, for the first time, enables direct control of a robotic hand mounted on the operator’s forearm. The system integrates kinesthetic force feedback and a universal motion mapping mechanism, and is built from off-the-shelf components combined with custom firmware and an open-source software stack, allowing seamless cross-platform deployment without structural modifications. User studies demonstrate an 86.67% task success rate, significantly outperforming conventional spatially separated teleoperation approaches while substantially reducing perceived operator workload.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Motion & Tracking

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Data-driven dexterous hand manipulation requires large-scale, physically consistent demonstration data. Simulation and video-based methods suffer from sim-to-real gaps and retargeting problems, while MoCap glove-based teleoperation systems require per-operator calibration and lack portability, as the robot hand is typically fixed to a stationary arm. Portable alternatives improve mobility but lack cross-platform and cross-operator compatibility. We present DEX-Mouse, a portable, calibration-free hand-held teleoperation interface with integrated kinesthetic force feedback, built from commercial off-the-shelf components under USD 150. The operator-agnostic design requires no calibration or structural modification, enabling immediate deployment across diverse environments and platforms. The interface supports a configuration in which the target robot hand is mounted directly on the forearm of an operator, producing robot-aligned data. In a comparative user study across various dexterous manipulation tasks, operators using the proposed system achieved an 86.67% task completion rate under the attached configuration. Also, we found that the attached configuration reduced the perceived workload of the operators compared to spatially separated teleoperation setups across all compared interfaces. The complete hardware and software stack, including bill of materials, CAD models, and firmware, is open-sourced at https://dex-mouse.github.io/ to facilitate replication and adoption.

Problem

• dexterous manipulation
• teleoperation
• data collection
• force feedback
• portable interface

Innovation

• teleoperation
• force feedback
• dexterous manipulation
• calibration-free
• portable interface