Tactile Genesis: Exploring Tactile Sensors at Scale for Learning Dexterous Tasks

2026-06-21

AI Summary

This study investigates how the type, layout, and resolution of tactile sensors influence performance and cost in dexterous manipulation. To this end, we develop a GPU-parallel simulation platform supporting multimodal tactile sensing—including contact, force/torque, elastomer displacement, audio, and temperature—augmented with realistic noise models. We introduce a voxelized temperature field to unify multimodal interfaces and enable efficient training of over 20,000 environments and 1,000 taxels on a single GPU. Through teacher-student policy distillation and real-to-sim transfer validation, we find that sensor coverage is far more critical than resolution: full-hand coverage significantly outperforms fingertip-only sensing, with force/torque information per taxel proving most practically valuable. The learned policies successfully transfer to the physical XHand1 platform.

Abstract

Tactile sensing is critical for contact-rich dexterous manipulation, yet it remains unclear which tactile abstractions a policy needs and when richer tactile fields justify their hardware cost. This is hard to study empirically: each sensor effectively defines a new robot, and no lab can replicate the same learning experiment across all of them. We present Tactile Genesis, a GPU-parallel tactile sensor simulation platform that exposes binary contact, contact depth, per-taxel kinematic force/torque, elastomer marker displacement, geometry-aware proximity, contact audio, and a voxelized temperature field (the first of its kind in robot learning physics simulation platforms) under a common interface, with configurable placement, resolution, and a realistic noise model (drift, hysteresis, dead taxels, crosstalk). It scales past 20,000 parallel environments and 1,000 taxels on a single GPU, improving throughput by 3 to 20 times over previous tactile simulators. We train teacher-student policies on three dexterous tasks, ablating sensor type, placement, resolution, and noise, and verify transfer to the real XHand1. Proprioception alone is insufficient on every task. Sensor placement dominates sensor type: fingertip-only coverage trails whole-hand coverage by a wide margin, while adding the palm and proximal phalanges closes most of the gap to the privileged teacher. Resolution matters far less than coverage: placing 200 taxels across the whole hand suffices across tasks. We find that force/torque per taxel is consistently the most useful sensor type. These results give concrete guidance for both future tactile hardware design for improving robot hands and policy-side observation choice in dexterous manipulation. https://neuroagents-lab.github.io/2026-tactile-genesis/

Problem

• tactile sensing
• dexterous manipulation
• sensor abstraction
• hardware cost
• robot learning

Innovation

• tactile simulation
• GPU-parallel
• dexterous manipulation
• sensor ablation
• real-to-sim transfer