ProxiDex: Learning Dynamics-Guided Proximity Policy for Dexterous Manipulation

2026-09-14

AI Summary

ProxiDex通过动态引导的邻近策略解决多指灵巧操作中手-物体交互部分可观测的问题，利用动作条件下的邻近动态学习来提高操作稳定性。

Abstract

Multi-finger dexterous manipulation relies on stable hand-object interactions, yet these interactions are partially observable in practice. Visual observations are often occluded by the hand, tactile sensors introduce hardware-specific modalities and calibration burdens, and existing policies rarely model how these cues evolve under actions, making them brittle under contact uncertainty. To address these, we present ProxiDex, a dynamics-guided proximity policy framework that treats hand-object proximity as an interaction state for dexterous manipulation. ProxiDex reconstructs interaction point clouds and converts geometric distances into proximity cues, forming a hardware-agnostic contact representation that provides immersive feedback during VR teleoperation. Built on this representation, ProxiDex learns action-conditioned proximity dynamics with a coupled forward-inverse design: future observation latents are predicted from actions, while proximity variations are decoded from latent changes. Leveraging these dynamics, ProxiDex adaptively reweights proximity tokens across manipulation phases and uses dynamics-consistency supervision to guide policy inference, stabilizing action generation under unreliable visual feedback. Simulation and real-world experiments demonstrate improved success rates and robustness over representative baselines across standard, unseen objects, and perturbation scenarios. Additional visualizations are available at https://proxidex.github.io/.

Problem

• multi-finger dexterous manipulation
• hand-object interactions
• partial observability
• contact uncertainty

Innovation

• dynamics-guided proximity policy
• interaction state
• hardware-agnostic contact representation
• action-conditioned proximity dynamics
• dynamics-consistency supervision