HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning

2026-02-27

AI Summary

This work addresses the challenge of sim-to-real transfer in contact-rich tasks, where existing tactile simulation methods fall short due to oversimplified modeling of shear forces and stick-slip dynamics. The authors propose a novel non-holonomic fluid-elastic tactile simulator based on Signed Distance Functions that, for the first time, jointly models stick-slip transitions, path-dependent shear force development, and full SE(3) object-sensor interactions. The framework supports arbitrary closed geometries and integrates seamlessly with multiple physics engines. Evaluated on the GelSight Mini sensor, the approach achieves high-fidelity shear signal simulation and demonstrates zero-shot transfer to four fine manipulation tasks with an average success rate of 93%, substantially outperforming methods based solely on tactile images (34%) and other shear simulation techniques (58%–61%).

Technology Category

• Intelligent Robots: Manipulation
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Computer Vision: Low Level & Physics-based Vision

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In this paper, we address the problem of tactile sim-to-real policy transfer for contact-rich tasks. Existing methods primarily focus on vision-based sensors and emphasize image rendering quality while providing overly simplistic models of force and shear. Consequently, these models exhibit a large sim-to-real gap for many dexterous tasks. Here, we present HydroShear, a non-holonomic hydroelastic tactile simulator that advances the state-of-the-art by modeling: a) stick-slip transitions, b) path-dependent force and shear build up, and c) full SE(3) object-sensor interactions. HydroShear extends hydroelastic contact models using Signed Distance Functions (SDFs) to track the displacements of the on-surface points of an indenter during physical interaction with the sensor membrane. Our approach generates physics-based, computationally efficient force fields from arbitrary watertight geometries while remaining agnostic to the underlying physics engine. In experiments with GelSight Minis, HydroShear more faithfully reproduces real tactile shear compared to existing methods. This fidelity enables zero-shot sim-to-real transfer of reinforcement learning policies across four tasks: peg insertion, bin packing, book shelving for insertion, and drawer pulling for fine gripper control under slip. Our method achieves a 93% average success rate, outperforming policies trained on tactile images (34%) and alternative shear simulation methods (58%-61%).

Problem

• tactile sim-to-real
• contact-rich tasks
• shear simulation
• policy transfer
• dexterous manipulation

Innovation

• Hydroelastic simulation
• Tactile sensing
• Sim-to-real transfer
• Stick-slip modeling
• Signed Distance Functions