ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models

2026-06-27

AI Summary

Existing embodied world models often suffer from trajectory drift and interaction distortion in long-horizon video generation due to the mismatch between low-dimensional actions and high-dimensional visual representations. This work proposes a dual-space collaborative modeling framework that unifies the visual space—incorporating pixel-aligned priors such as end-effector poses, depth geometry, and robot masks—with the parameter space comprising raw action sequences and camera matrices. By explicitly integrating geometric constraints with numerically driven actions, the approach achieves unprecedented consistency in long-term trajectory prediction and precision in action execution. It enables robust interaction with complex deformable objects, such as cloth folding, and demonstrates strong generalization under out-of-distribution and cross-embodiment settings, while remaining compatible with diverse backbone architectures.

Abstract

Embodied World Models (EWMs) have emerged as a scalable and risk-free paradigm for advancing embodied intelligence, enabling the safety-critical evaluation of Vision-Language-Action systems. However, their reliability as evaluation benchmarks and foundational simulators is often hindered by the representation gap between low-dimensional actions and high-dimensional video synthesis. This gap results in a lack of geometric correspondence, manifesting as accumulated trajectory drift and inconsistent robot-object interactions during long-horizon rollouts. To bridge this gap, we propose ViPSim, a framework that achieves consistent long-horizon generation through the synergistic collaboration of Visual and Parameter Spaces. We define the Visual Space as a domain of explicit spatial priors, integrating pixel-aligned projections of end-effector pose, camera perspectives, depth-informed scene geometry, and robotic morphological masks to provide dense structural grounding. Concurrently, the Parameter Space serves as a domain of numerical drivers, injecting raw action sequences and camera matrices to provide precise motion guidance. By unifying these two spaces, ViPSim ensures that the generated states are simultaneously anchored by geometric boundaries and steered by numerical commands. Extensive experiments demonstrate that ViPSim is backbone-agnostic and significantly enhances trajectory consistency. Notably, our approach exhibits emergent capabilities in generating complex interactions with deformable objects (e.g., cloth folding) and maintains robust performance in out-of-distribution and cross-embodiment scenarios, providing a high-fidelity foundation for the automated evaluation and predictive control of embodied agents.

Problem

• Embodied World Models
• representation gap
• trajectory drift
• robot-object interaction
• long-horizon simulation

Innovation

• Embodied World Models
• Visual-Parameter Collaboration
• Long-Horizon Consistency
• Geometric Grounding
• Deformable Object Interaction