DexTeleop-0: Force-Aware Bimanual Dexterous Teleoperation with Ego-Centric Perception towards Shared Autonomy

2026-06-22

AI Summary

Traditional teleoperation systems struggle with contact-intensive bimanual dexterous manipulation due to embodiment mismatches and the absence of tactile and force feedback, limiting data collection efficiency for high-precision tasks. This work proposes DexTeleop-0, a bimanual dexterous teleoperation framework that introduces a novel tactile-driven adaptive strategy: by estimating contact points and sensing fingertip forces, it dynamically optimizes joint commands through operational-space Jacobian refinement, translating coarse human motions into fine-grained robotic control compliant with contact forces. Integrating tactile sensing, force feedback, egocentric perception, and a shared autonomy architecture, the system substantially outperforms baseline methods in both simulation and real hardware, achieving higher success rates and execution efficiency in robust grasping, disturbance-resistant manipulation, and complex dexterous tasks.

Abstract

Fine-grained, bimanual dexterous manipulation remains a foundational challenge in robotics. Traditional teleoperation systems often fail in contact-rich tasks because embodiment gaps hinder accurate kinematic mapping, while tactile and force feedback remain absent. Consequently, data collection efficiency for high-precision tasks remains prohibitively low. To address these limitations, we propose a tactile-driven adaptation strategy designed to enable fine-grained manipulation on top of teleoperation pipelines. Instantiated within our bimanual dexterous framework, DexTeleop-0, this strategy introduces a real-time optimization loop that bridges the embodiment gap by translating coarse human tracking intents into precise, force-compliant robotic commands with tactile sensing. By estimating accurate contact points and leveraging a tactile-enabled fingertip force-sensing profile, the system dynamically computes localized corrections using the operational space Jacobian with respect to joint angle updates. We rigorously evaluate this tactile-driven adaptation strategy across both simulated environments and real-world hardware. Compared with representative baselines, the proposed method consistently achieves higher task success rates and improved execution efficiency in robust grasping, disturbance-resilient manipulation, and complex dexterous tasks.

Problem

• dexterous manipulation
• teleoperation
• force feedback
• tactile sensing
• embodiment gap

Innovation

• tactile-driven adaptation
• bimanual dexterous teleoperation
• force-aware control
• embodiment gap bridging
• operational space Jacobian