Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention

2026-06-25

AI Summary

In contact-rich robotic manipulation, relying solely on vision often fails to accurately predict critical tactile states such as slipping or sticking, leading to ineffective action decisions. To address this limitation, this work proposes a tactile-aware world model for action generation, introducing a novel Tactile Asymmetric Attention Mechanism (TAAM). The approach employs a VideoClean mask to prevent tactile information from interfering with visual prediction and incorporates a tactile perceptual bias to integrate predicted tactile changes into action synthesis. This design effectively decouples the distinct influence pathways of tactile feedback on visual perception and motor control. Evaluated on the ManiFeel benchmark, the method achieves a 38.9% absolute improvement in overall task success rate, with performance gains reaching up to 86% in high-contact scenarios.

Abstract

World Action Models (WAMs) generate actions together with predicted futures, offering a powerful interface for robot decision making. In contact-rich manipulation, however, visually plausible futures can be physically incomplete: insertion, assembly, search, and reorientation often depend on slip, jamming, contact normals, or small alignment errors that are weakly visible or hidden in RGB. A natural solution is to predict future tactile states, however, we identify tactile pollution, a failure mode where unconstrained tactile-token injection degrades video and action prediction by forcing a visual dynamics model to absorb sparse, local, event-driven contact signals. To address this, we propose Tactile-WAM, a touch-aware WAM with a Tactile Asymmetric Attention Mechanism (TAAM). TAAM combines a VideoClean mask, which blocks video-query access to tactile key/value tokens while preserving action-query access, with a touch-aware bias for action attention. The VideoClean mask protects visual prediction while keeping contact information available for action generation; the touch-aware bias is derived from predicted touch changes and modulates action attention to tactile tokens during denoising. On ManiFeel, Tactile-WAM improves the mean success rate by 38.9% overall and by 86% on contact-rich tasks.

Problem

• tactile perception
• world action models
• contact-rich manipulation
• tactile pollution
• robot decision making

Innovation

• Tactile-WAM
• Tactile Asymmetric Attention
• World Action Model
• tactile pollution
• contact-rich manipulation