You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact

2026-06-27

AI Summary

This work addresses the challenge of six-degree-of-freedom (6-DoF) pose estimation in visually degraded scenarios—such as occlusion, low illumination, or reflective/transparent objects—by proposing a method that recovers full 6-DoF pose from just two tactile contacts. Leveraging a local 3D tactile point cloud representation, the approach employs a coarse-to-fine network to accurately localize contact points and introduces a normal-aware closed-form SVD solver to directly compute the object pose in a single step. Notably, the method requires no contact history, is compatible with object models reconstructed from consumer-grade scanners, and utilizes a virtual-to-real domain fine-tuning strategy. Experiments on four geometrically diverse objects demonstrate significantly higher pose accuracy compared to visual and geometry-based baselines, particularly excelling under conditions where visual cues are unreliable.

Abstract

Accurate 6-DoF object pose estimation is fundamental to robotic manipulation, yet vision-based methods often fail under occlusion, poor lighting, and reflective or transparent surfaces. We present YOTO, a tactile-only pose estimation system that recovers the full 6-DoF object pose from a single pair of simultaneous contacts, without requiring contact history. YOTO represents each tactile contact as a local 3D point cloud and localizes it on the object surface through a coarse-to-fine network. The two localized contacts, together with the calibrated sensor poses, are then fed to a closed-form normal-aware SVD solver that recovers the full 6-DoF object pose in one step. To reduce real-data requirements, the localization network is pretrained on virtual tactile patches sampled from the object model and fine-tuned with a small number of real contacts. We further show that YOTO can operate on object models reconstructed from consumer-grade mobile scans, and quantify the gap relative to CAD-based models. Experiments on four geometrically diverse objects demonstrate accurate tactile contact localization and pose estimation, outperforming vision-based and geometric baselines, especially when visual perception is unreliable. Code, trained models, and the real GelSight dataset will be released upon publication.

Problem

• 6-DoF object pose estimation
• tactile sensing
• robotic manipulation
• occlusion
• vision failure

Innovation

• tactile pose estimation
• 6-DoF
• single contact
• normal-aware SVD
• virtual-to-real transfer