ContactWorld: What Matters in Vision-Tactile World Models for Contact-Rich Manipulation

2026-06-11

AI Summary

This work addresses the challenge of building vision–tactile world models capable of supporting contact-intensive manipulation tasks for stable, long-horizon robotic planning. To this end, the authors introduce ContactWorld, a benchmark comprising twelve tasks designed to systematically evaluate the performance of diverse world models. Their analysis reveals that spatially structured and temporally coherent representations are critical for effective planning, and that tactile benefits hinge on cross-modal representational compatibility rather than merely increasing modality scale. Experimental results demonstrate that incorporating point cloud observations boosts average planning success rates from 20.7%–22.0% to 32.1%, and further integration of tactile force field representations elevates performance to 36.1%, substantially outperforming existing approaches.

Abstract

Contact-rich manipulation requires world models to reason over complex contact dynamics from multimodal sensory observations. However, it remains unclear which representation properties fundamentally support stable long-horizon planning in contact-rich settings. In this paper, we present ContactWorld, a benchmark and systematic empirical study of vision-tactile world models spanning 12 contact-rich manipulation tasks, including insertion, disassembly, screwing, and exploratory interaction. Across extensive experiments, we find that representations that are both spatially structured and temporally continuous consistently achieve the strongest planning performance. In particular, point-cloud observations improve average planning success rates from 20.7% with wrist-view observations and 22.0% with front-view observations to 32.1%. We further find that the effectiveness of tactile sensing depends critically on cross-modal representation compatibility rather than modality scaling alone. Combining point-cloud observations with tactile force-field representations, which preserve richer spatial structure and interaction dynamics, further improves performance to 36.1%, yielding the strongest overall planning performance across all evaluated tasks. Moreover, tactile sensing becomes increasingly important under long-horizon planning objectives, where compounding prediction errors and contact uncertainty accumulate over time. Together, these findings highlight the importance of representation structure, multimodal compatibility, and long-horizon robustness in vision-tactile world models for contact-rich robotic manipulation.

Problem

• contact-rich manipulation
• world models
• vision-tactile perception
• long-horizon planning
• representation structure

Innovation

• vision-tactile world models
• spatially structured representations
• tactile force-field
• contact-rich manipulation
• long-horizon planning