Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catching

2026-03-30

AI Summary

Existing teleoperation methods often fail in dexterous grasping of dynamically moving objects due to errors in timing, pose estimation, and force control. This work proposes Tele-Catch, a framework that integrates human teleoperation signals with autonomous policies to achieve robust grasping of dynamic 3D objects. The key innovations include a Dynamics-Aware Adaptive Shared Control mechanism (DAIM) that dynamically adjusts the control authority between human and robot based on real-time object states, and a diffusion-policy-based denoising grasp generation module coupled with DP-U3R—an unsupervised point cloud geometric representation method—to enhance policy generalization. Experimental results demonstrate that Tele-Catch significantly improves both success rate and robustness across multiple dexterous hand platforms and unseen object categories.

Abstract

Teleoperation is a key paradigm for transferring human dexterity to robots, yet most prior work targets objects that are initially static, such as grasping or manipulation. Dynamic object catch, where objects move before contact, remains underexplored. Pure teleoperation in this task often fails due to timing, pose, and force errors, highlighting the need for shared autonomy that combines human input with autonomous policies. To this end, we present Tele-Catch, a systematic framework for dexterous hand teleoperation in dynamic object catching. At its core, we design DAIM, a dynamics-aware adaptive integration mechanism that realizes shared autonomy by fusing glove-based teleoperation signals into the diffusion policy denoising process. It adaptively modulates control based on the interaction object state. To improve policy robustness, we introduce DP-U3R, which integrates unsupervised geometric representations from point cloud observations into diffusion policy learning, enabling geometry-aware decision making. Extensive experiments demonstrate that Tele-Catch significantly improves accuracy and robustness in dynamic catching tasks, while also exhibiting consistent gains across distinct dexterous hand embodiments and previously unseen object categories.

Problem

• teleoperation
• dynamic object catching
• dexterous manipulation
• shared autonomy
• 3D object interaction

Innovation

• shared autonomy
• diffusion policy
• dynamic object catching
• adaptive teleoperation
• geometry-aware learning