UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models

2026-06-30

AI Summary

Existing vision-language-action models struggle to effectively model tactile semantics and future physical interactions in contact-rich dexterous manipulation. This work proposes a unified tactile latent space that, for the first time, treats tactile signals as dynamic interaction cues to construct a state- and dynamics-aware tactile prior. By integrating chain-of-thought tactile reasoning with coarse-to-fine future tactile prediction, the approach jointly models current states and anticipated contact changes. A hybrid controller is further designed to fuse real-time and predicted tactile feedback. Evaluated on four representative tasks—adjustment, insertion, wiping, and assembly—the method significantly outperforms existing approaches in success rate, manipulation accuracy, and contact robustness.

Abstract

Vision-language-action (VLA) models have achieved strong performance in many robotic manipulation tasks, yet remain limited in contact-rich dexterous manipulation. To overcome this limitation, recent vision-tactile-language-action (VTLA) methods incorporate tactile sensing into VLA models to provide direct contact information. However, they typically treat tactile signals as passive auxiliary inputs, making it difficult to model tactile semantics and future physical interactions. To this end, we propose a unified tactile learning framework for contact-rich manipulation that models tactile signals as dynamic interaction cues for both contact understanding and prediction. Specifically, we construct a unified tactile latent space and jointly model current tactile states and future contact changes through tactile chain-of-thought reasoning and coarse-to-fine future tactile prediction, thereby forming a state-aware and dynamics-aware tactile prior. Based on this prior, we introduce a tactile-action mixed controller that combines real-time and predicted tactile feedback to refine low-frequency action chunks with high-frequency corrections. Real-world experiments on four categories of contact-rich tasks, including adjustment, insertion, wiping, and assembly, under both clean and externally perturbed settings, show that our method improves success rate, manipulation accuracy, and contact robustness over existing methods, demonstrating its effectiveness in dexterous physical interaction.

Problem

• dexterous manipulation
• tactile sensing
• contact-rich tasks
• vision-language-action models
• physical interaction

Innovation

• tactile understanding
• tactile prediction
• vision-tactile-language-action
• chain-of-thought reasoning
• tactile-action control