Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulation

2026-07-01

AI Summary

Existing vision-language-action models struggle to effectively integrate tactile modality for modeling contact dynamics, and the limited scale and narrow scope of current tactile datasets hinder dexterous manipulation performance. To address this, this work introduces H-Tac, a large-scale tactile-action dataset comprising 160 hours of first-person human videos, and proposes TTP, a transferable tactile pretraining framework. TTP unifies human and robotic tactile-action representation spaces, explicitly models contact dynamics, and incorporates a tactile-expert-guided future prediction mechanism, enabling cross-domain knowledge transfer from large-scale human tactile data for the first time. Experiments demonstrate that TTP significantly enhances generalization and fine-grained control in dexterous manipulation tasks on both simulated and real robots.

Abstract

As an essential modality for dexterous and contact-rich tasks, tactile sensing provides precise force feedback that cannot be reliably inferred from vision. However, limited by hardware and data collection systems, existing datasets with tactility remain small in scale and narrow in contact coverage. Meanwhile, Vision-Language-Action (VLA) models with tactile modality are constrained on dynamics-agnostic post-training, which limits the performance ceiling on downstream tasks. In this paper, we present H-Tac, a large-scale tactile-action dataset with 160-hour egocentric human videos containing more than 300 tasks and 135k episodes. Building upon this, we propose Transferable Tactile Pre-Training (TTP), a system of tactile-based pre-training on human data for fine-grained robotic tasks. To bridge the gap between humans and robots, we use unified tactile and action spaces throughout the pre-training and post-training phases, preserving prior knowledge during human-to-robot transfer. By leveraging a tactile expert for future tactile prediction, our framework explicitly models the contact dynamics and precise physical interactions. Extensive experiments in simulation and on real robots demonstrate that our model achieves superior performance, exhibiting robust generalization and fine-grained manipulation capabilities. TTP paves the way for scalable tactile pre-training via human-to-robot transfer.

Problem

• tactile sensing
• dexterous manipulation
• human-to-robot transfer
• contact dynamics
• pre-training

Innovation

• tactile pre-training
• human-to-robot transfer
• contact dynamics modeling
• large-scale tactile dataset
• dexterous manipulation