T-Rex: Tactile-Reactive Dexterous Manipulation

2026-06-15

AI Summary

This work addresses the limitation of existing vision–language–action (VLA) models in robotic dexterous manipulation, which typically neglect tactile feedback or rely solely on static tactile representations, thereby failing to support dynamic tactile responses. To overcome this, the study introduces dynamic tactile perception into the VLA framework for the first time, proposing three core innovations: a large-scale, motion-primitive-based dataset enriched with high-frequency tactile signals, a temporal tactile VQ-VAE encoder that captures time-varying tactile features, and a variable-rate Mixture-of-Transformers architecture. The proposed method effectively leverages rich tactile dynamics, achieving an average success rate improvement of over 30% compared to the strongest baseline across twelve fine-grained force-control and deformable object manipulation tasks.

Abstract

The ability to react dynamically to tactile signals has long been considered crucial to agile human-level dexterity. Yet contemporary learning-based Vision-Language-Action (VLA) models for robotic manipulation generally either overlook the tactile modality or are limited to encoders with static cues, due in part to the scarcity of diverse training data and standardized evaluation, architectural constraints in current VLA models, and limitations of static tactile encoders. In this paper, we push the frontier of tactile-reactive manipulation by addressing all of these limitations. We propose a large-scale, 100-hour tactile-rich dataset collected via a novel, data-efficient recipe that prioritizes elementary motor primitives. To effectively exploit naturally high-frequency touch signals without sacrificing the existing capabilities of existing VLAs, we introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder. We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation, achieving over 30% higher average success rate than the strongest baseline.

Problem

• tactile-reactive manipulation
• dexterous manipulation
• Vision-Language-Action models
• tactile modality
• robotic manipulation

Innovation

• tactile-reactive manipulation
• Mixture-of-Transformers
• temporal tactile VQ-VAE
• dexterous manipulation
• Vision-Language-Action models