TaCauchy: An Extensible FEM Framework for Vision-Based Tactile Simulation

2026-06-18

AI Summary

Existing approaches struggle to achieve high-fidelity, physics-based simulation of visual-tactile stress fields on GPU-accelerated robotic platforms, thereby limiting force-aware training in reinforcement learning. This work proposes a scalable finite element framework integrated into Isaac Sim that, for the first time, combines a unified incremental potential contact (UIPC) solver with automatic geometry-aware mesh refinement to compute Cauchy stress tensors directly from hyperelastic constitutive models and project them onto contact surfaces, enabling first-principles-driven tactile mechanics simulation. The method supports plug-and-play compatibility with diverse vision-based tactile sensors, achieving sub-millisecond stress extraction latency at 33.40 FPS in a single environment and a total throughput of 555 FPS across 60 parallel environments. Within a force range of 1.26–4.73 N, simulated tactile images exhibit structural similarity (SSIM) exceeding 0.93 compared to real-world measurements, demonstrating high physical fidelity.

Abstract

Vision-based tactile sensors require high-fidelity simulation for reinforcement learning, yet existing approaches struggle to provide accurate mechanical stress fields within GPU-accelerated robotics platforms. We present TaCauchy, an extensible Finite Element Method (FEM) framework that integrates rigorous physics-based force computation into Isaac Sim. Built on the Unified Incremental Potential Contact (UIPC) solver, TaCauchy directly computes Cauchy stress tensors from hyperelastic constitutive laws and projects them onto contact surfaces to obtain traction forces and pressure distributions, providing mechanical ground truth from first principles rather than empirical estimation. Our framework features automatic mesh generation with geometry-aware adaptive refinement and a modular sensor interface enabling rapid integration of diverse sensors (GelSight Mini, DIGIT, 9DTact) with minimal configuration. Performance benchmarks demonstrate 33.40 FPS for single environments and 555 FPS aggregate throughput across 60 parallel environments, with stress extraction overhead under 1 ms. Physical validation experiments show strong agreement between simulated and real tactile responses across force ranges from 1.2556 N to 4.7332 N, achieving SSIM above 0.93, confirming the framework's capability to provide accurate, physically-grounded force supervision for downstream robotic manipulation tasks.

Problem

• vision-based tactile simulation
• mechanical stress field
• GPU-accelerated robotics
• force supervision
• high-fidelity simulation

Innovation

• Finite Element Method
• Cauchy stress tensor
• vision-based tactile simulation
• GPU-accelerated robotics
• modular sensor integration