Imagining the Sense of Touch: Touch-Informed Manipulation via Imagined Tactile Representations

2026-07-02

AI Summary

This work addresses the challenge of enhancing robotic performance in contact-intensive tasks without relying on physical tactile sensors. The authors propose TacImag, a framework that predicts tactile signals—such as force fields or tactile images—from visual and proprioceptive inputs, using these predictions as supervisory signals for policy learning. TacImag is the first approach to enable tactile-augmented manipulation control without deploying actual tactile hardware, demonstrating that the core benefit of tactile imagination lies not in faithfully reconstructing tactile data but in providing contact-aware representations that facilitate policy optimization. Experiments across six simulated and four real-world tasks show substantial performance gains: force-field representations improve contact-sensitive tasks by 44.4% on average, while tactile-image representations boost texture-sensitive tasks by 23.3%.

Abstract

Tactile sensing can substantially improve contact-rich robotic manipulation, yet its practical deployment remains limited by the fragility, calibration requirements, and maintenance burden of tactile hardware. This raises a fundamental question: can robots benefit from tactile knowledge without requiring tactile sensors at deployment? We present TacImag, a tactile imagination framework that predicts tactile observations from vision and proprioception and uses the generated signals to guide manipulation policies. Trained from paired visuotactile demonstrations, TacImag enables touch-informed manipulation using only visual observations at test time. We evaluate TacImag on six simulated and four real-world manipulation tasks. Across simulation and real-world experiments, imagined tactile observations consistently improve manipulation performance without requiring tactile hardware. In real-world experiments, imagined force fields improve contact-sensitive tasks by 44.4% on average, whereas imagined tactile images improve texture-sensitive tasks by 23.3%, revealing that the effectiveness of tactile imagination depends strongly on the relationship between tactile representation and task requirements. Our results further suggest that tactile imagination does not simply recover missing tactile measurements. Instead, it acts as a form of contact-aware supervision that transforms subtle visual interaction cues into representations that are easier for manipulation policies to exploit.

Problem

• tactile sensing
• robotic manipulation
• sensorless touch
• tactile imagination
• contact-rich tasks

Innovation

• tactile imagination
• vision-based tactile prediction
• touch-informed manipulation
• sensorless tactile sensing
• contact-aware supervision