Geometric Reconstruction of Extrinsic Contact Trajectories using Tactile Sensing and Proprioception for Tool Manipulation

2026-06-20

AI Summary

This work addresses the challenge of reconstructing tool-tip contact trajectories when tactile sensors are located far from the point of interaction, making direct perception infeasible. Framing the problem as a geometric reasoning task under a single-point contact assumption, the method fuses tactile and proprioceptive data to first estimate the global contact location via a calibration phase and then reconstructs the full trajectory using sequential tactile marker observations. To the best of our knowledge, this is the first approach to achieve stable and robust tool-tip trajectory estimation using only grasp-level tactile information, demonstrating adaptability across diverse tool types, wrist poses, and grasp configurations. Experimental results show a trajectory root-mean-square error (RMSE) of 8.59 ± 2.41 mm and a shape RMSE of 5.96 ± 1.16 mm in world coordinates, with an online processing rate of 14.00 ± 4.11 Hz.

Abstract

Tactile sensing enables robots to perceive rich contact information at the grasp, supporting tasks such as object recognition, in-hand pose estimation, and slip detection. However, in many tool-mediated manipulation tasks, the interaction that determines task success occurs at the tool tip, away from the tactile sensor, making direct sensing of tool-environment contact difficult, particularly when the contact moves during interaction. In this work, we reconstruct the trajectory of extrinsic tool-tip contact using tactile sensing and robot proprioception. We formulate tool-tip trajectory reconstruction as a geometric inference problem under a single-point contact assumption. Our method first estimates the global tool-tip contact location from a calibration segment designed to approximate fixed-point behavior, and then reconstructs the full trajectory by composing relative tool motion estimated from tactile marker observations under continuous contact. Across n=51 trials with multiple trajectories, tools, wrist poses, and grasp configurations, the proposed pipeline achieves a trajectory RMSE of 8.59 +/- 2.41 mm in the world frame and a shape RMSE of 5.96 +/- 1.16 mm, while operating online at 14.00 +/- 4.11 Hz. Overall, the results show that extrinsic tool-tip trajectory geometry can be recovered consistently from grasp-level tactile sensing, with trajectory shape remaining stable across variations in tools, wrist poses, and grasp configurations.

Problem

• tactile sensing
• tool manipulation
• contact trajectory
• geometric reconstruction
• proprioception

Innovation

• tactile sensing
• geometric reconstruction
• tool manipulation
• proprioception
• contact trajectory