SPLIT: Separating Physical-Contact via Latent Arithmetic in Image-Based Tactile Sensors

2026-04-27

AI Summary

This work addresses the scarcity of real-world tactile interaction data, which hinders the training and generalization of robotic tactile perception models. To overcome this limitation, the authors propose SPLIT, a novel method that, for the first time, explicitly disentangles contact geometry from sensor-specific optical properties in a latent space through representation learning. This enables efficient simulation and cross-device transfer for image-based tactile sensors. By integrating finite-element soft-body mesh simulations with variable-resolution FEM, SPLIT supports bidirectional mapping—generating tactile images from surface deformations and reconstructing deformations from tactile images—without requiring retraining. Experiments demonstrate that SPLIT achieves faster inference than existing approaches and successfully transfers to heterogeneous sensors such as DIGIT and GelSight R1.5, offering a high-fidelity, tunable tactile simulation environment.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Biometrics, Face, Gesture & Pose
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Data transparency and provenance

Abstract

Training machine learning models for robotic tactile sensing requires vast amounts of data, yet obtaining realistic interaction data remains a challenge due to physical complexity and variability. Simulating tactile sensors is thus a crucial step in accelerating progress. This paper presents SPLIT, a novel method for simulating image-based tactile sensors, with a primary focus on the DIGIT sensor. Central to our approach is a latent space arithmetic strategy that explicitly disentangles contact geometry from sensor-specific optical properties. Unlike methods that require recalibration for every new unit, this disentanglement allows SPLIT to adapt to diverse DIGIT backgrounds and even transfer data to distinct sensors like the GelSight R1.5 without full model retraining. Beyond this adaptability, our approach achieves faster inference speeds than existing alternatives. Furthermore, we provide a calibrated finite element method (FEM) soft-body mesh simulation with variable resolution, offering a tunable trade-off between speed and fidelity. Additionally, our algorithm supports bidirectional simulation, allowing for both the generation of realistic images from deformation meshes and the reconstruction of meshes from tactile images. This versatility makes SPLIT a valuable tool for accelerating progress in robotic tactile sensing research.

Problem

• tactile sensing
• data scarcity
• physical contact
• simulation
• robotic perception

Innovation

• latent space arithmetic
• tactile sensor simulation
• disentangled representation
• bidirectional simulation
• cross-sensor transfer