GelSLAM: A Real-time, High-Fidelity, and Robust 3D Tactile SLAM System

2025-08-21

AI Summary

To address the challenges of long-term object pose estimation and high-fidelity shape reconstruction under visually constrained conditions, this paper proposes the first purely tactile-driven real-time 3D SLAM system. Methodologically, it innovatively introduces tactile-derived surface normals and curvature as geometric priors—replacing conventional point-cloud registration—to enable robust pose tracking and loop closure detection. By integrating high-resolution tactile sensing, differential geometric analysis, and an optimized SLAM framework, the system establishes a continuous joint morphology-motion model. Experimental results demonstrate sub-millimeter (<0.5 mm) shape reconstruction accuracy and low-drift global pose estimation during in-hand manipulation tasks, significantly outperforming existing tactile SLAM approaches. Notably, the system maintains stable performance on low-texture objects (e.g., wooden tools), where visual methods typically fail.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Low Level & Physics-based Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Accurately perceiving an object's pose and shape is essential for precise grasping and manipulation. Compared to common vision-based methods, tactile sensing offers advantages in precision and immunity to occlusion when tracking and reconstructing objects in contact. This makes it particularly valuable for in-hand and other high-precision manipulation tasks. In this work, we present GelSLAM, a real-time 3D SLAM system that relies solely on tactile sensing to estimate object pose over long periods and reconstruct object shapes with high fidelity. Unlike traditional point cloud-based approaches, GelSLAM uses tactile-derived surface normals and curvatures for robust tracking and loop closure. It can track object motion in real time with low error and minimal drift, and reconstruct shapes with submillimeter accuracy, even for low-texture objects such as wooden tools. GelSLAM extends tactile sensing beyond local contact to enable global, long-horizon spatial perception, and we believe it will serve as a foundation for many precise manipulation tasks involving interaction with objects in hand. The video demo is available on our website: https://joehjhuang.github.io/gelslam.

Problem

• Accurately perceive object pose and shape
• Track and reconstruct objects using tactile sensing
• Enable global spatial perception for precise manipulation

Innovation

• Uses tactile-derived surface normals
• Employs curvature data for robust tracking
• Enables real-time submillimeter accuracy reconstruction