HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation

2026-08-19

AI Summary

该研究提出HiTac-WAM模型，通过分层预测触觉状态解决机器人在接触丰富操作中的未来触觉建模问题，提高任务成功率。

Abstract

World action models jointly predict future visual observations and actions, whereas existing tactile-aware variants typically represent future touch as an image or latent stream without modeling the physical dependencies that organize tactile states hierarchically. We present HiTac-WAM, a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution. The forecast factorizes into contact state, a 3D deformation field, and slip risk, organized as a directed hierarchy in which each downstream stage is conditioned on stop-gradient signals from preceding stages. A directed attention mask allows tactile queries to attend to the video-action context of each candidate while preventing video and action queries from attending to tactile tokens. For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates. For execution, the selected tactile forecast is retained as a reference; persistent discrepancies between predicted and observed tactile states trigger corrective replanning. HiTac-WAM achieves a mean contact F1 of 0.921; under matched training budgets, the directed hierarchy reduces 3D displacement L2 error by 17.6% relative to the deformation-only predictor and improves slip AUPRC by 60.4% relative to the slip-only predictor. Across chip grasping, blackboard erasing, and USB insertion, selection guided by the hierarchical forecasts increases the average real-robot success rate from 31.1% to 61.1%, while the full system attains 72.2%.

Problem

• tactile states
• hierarchical organization
• robot manipulation
• physical dependencies
• future prediction

Innovation

• Hierarchical Tactile World Action Model
• Directed Hierarchy
• Tactile Forecasting
• Stop-Gradient Signals
• Directed Attention Mask