DexEMG: Towards Dexterous Teleoperation System via EMG2Pose Generalization

2026-03-06

AI Summary

This work addresses the longstanding trade-off between performance and portability in dexterous hand teleoperation systems, where vision-based approaches suffer from occlusion and high cost, while exoskeleton devices are cumbersome and restrict natural hand motion. The authors propose a lightweight sEMG-based teleoperation framework that leverages a neural network, EMG2Pose, to continuously estimate hand poses from surface electromyography (sEMG) signals, coupled with a real-time hand retargeting algorithm to drive a multi-fingered dexterous hand. Requiring minimal user-specific calibration, the system demonstrates high-precision control across users, objects, and complex environments, exhibiting strong generalization and practicality. This approach offers an efficient and versatile solution for general-purpose robotic manipulation and assistive technologies.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Machine Learning: Learning with Manifolds

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

High-fidelity teleoperation of dexterous robotic hands is essential for bringing robots into unstructured domestic environments. However, existing teleoperation systems often face a trade-off between performance and portability: vision-based capture systems are constrained by costs and line-of-sight requirements, while mechanical exoskeletons are bulky and physically restrictive. In this paper, we present DexEMG, a lightweight and cost-effective teleoperation system leveraging surface electromyography (sEMG) to bridge the gap between human intent and robotic execution. We first collect a synchronized dataset of sEMG signals and hand poses via a MoCap glove to train EMG2Pose, a neural network capable of continuously predicting hand kinematics directly from muscle activity. To ensure seamless control, we develop a robust hand retargeting algorithm that maps the predicted poses onto a multi-fingered dexterous hand in real-time. Experimental results demonstrate that DexEMG achieves high precision in diverse teleoperation tasks. Notably, our system exhibits strong generalization capabilities across novel objects and complex environments without the need for intensive individual-specific recalibration. This work offers a scalable and intuitive interface for both general-purpose robotic manipulation and assistive technologies.

Problem

• teleoperation
• dexterous robotic hands
• EMG-based control
• human-robot interface
• portability-performance trade-off

Innovation

• sEMG
• EMG2Pose
• teleoperation
• hand retargeting
• generalization