TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training

2026-07-02

AI Summary

This work addresses the susceptibility of vision-language-action (VLA) models to unrecoverable failures in contact-intensive manipulation tasks due to minor perturbations. To mitigate this issue, the authors propose TACO, a framework that integrates a tactile-aware world model to generate contact-consistent visuo-tactile corrective trajectories through a Recognize-Imagine-Label loop. TACO further introduces a knowledge-isolated tactile adaptation mechanism that injects tactile error-correction supervision while preserving pretrained vision-language priors, enabling efficient post-training via advantage-conditioned learning. Experimental results demonstrate that TACO achieves an absolute success rate improvement of 44% over baseline policies and outperforms tactile adaptation methods without knowledge isolation by 32% on real-world contact-intensive tasks.

Abstract

Vision-Language-Action (VLA) models have shown promising generalization in robotic manipulation, but they still struggle with contact-rich tasks, where minor contact perturbations can cause unrecoverable failures that are hard to detect from vision alone. Since these failures are localized rather than task-level semantic errors, tactile-aware corrective post-training offers an efficient way to improve recovery. However, scaling such supervision through human intervention is costly. Recent works have explored world models to synthesize imagined rollouts for policy improvement, but vision-only world models may produce visually plausible yet contact-inconsistent trajectories. We therefore introduce TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation. Given real robot rollouts, TACO follows a Recognize-Imagine-Label loop with a tactile-aware world model: a unified progress-action model recognizes failure-adjacent states using progress estimates, a visuo-tactile generation model imagines local correction segments, and the progress-action model labels them with executable corrective actions. To incorporate tactile corrective supervision into VLA post-training, TACO combines knowledge-insulated tactile adaptation with advantage-conditioned training, enabling the policy to learn from imagined corrections without degrading pretrained visual-language priors. These components enable TACO to convert real-world failures into imagined visuo-tactile corrections for iterative VLA post-training. Experiments on real-world contact-rich manipulation tasks show that TACO achieves 44% absolute success rate improvement over the base policy and 32% over the policy without knowledge-insulated tactile adaptation.

Problem

• Vision-Language-Action
• contact-rich manipulation
• tactile feedback
• policy post-training
• failure recovery

Innovation

• tactile-aware world model
• VLA post-training
• knowledge-insulated adaptation
• visuo-tactile generation
• self-correction