VISTA: Visually Inferred Spatial ConTact Attention for Contact-Rich Manipulation

2026-08-26

AI Summary

为解决接触丰富操作中精确交互反馈的问题，VISTA-Policy利用视觉变形场作为高维视觉-物理反馈，结合物理感知编码引擎、能量聚集去噪机制和形变增强策略网络方法。

Abstract

Contact-rich manipulation requires precise interaction feedback. While vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object--gripper interactions; dedicated tactile or force sensors can provide rich contact information but introduce additional hardware complexity, calibration requirements, and deployment costs. To bridge this gap, we propose VISTA-Policy, an imitation learning paradigm that utilizes the Visual Deformation Field (VDF), a 3D displacement representation of a compliant gripper, as high-dimensional visuo-physical feedback. The framework integrates: 1) a Physics-Aware Encoding Engine for real-time VDF decoding; 2) an Energy Aggregation Denoising Mechanism to isolate true interaction signals; and 3) a Deformation-Augmented Policy Network with incremental gripper actions for precise closed-loop correction. Extensive evaluations on Cross-Scale Object Grasping, Cap Unscrewing, and Calligraphy Writing demonstrate that VISTA-Policy outperforms the strong pure-vision baseline 3D Diffusion Policy and the tactile baseline. VISTA-Policy further demonstrates substantial out-of-distribution generalization to unseen object scales and robustness against dynamic disturbances, offering a durable and cost-effective route toward general-purpose fine-grained manipulation in unstructured environments. Project videos and supplementary materials are available at: https://sites.google.com/view/vista-policy.

Problem

• contact-rich manipulation
• interaction feedback
• visual observation
• tactile sensors
• hardware complexity

Innovation

• Visually Inferred Spatial ConTact Attention
• Visual Deformation Field (VDF)
• Physics-Aware Encoding Engine
• Energy Aggregation Denoising Mechanism
• Deformation-Augmented Policy Network