TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image

2026-09-04

AI Summary

TourPhysics通过结合物理模拟与视频生成，从单张图片实现持续探索和操作，解决当前视频世界模型在长时序下物理或空间一致性丢失的问题。

Abstract

Interactive visual world models must distinguish observation from physical intervention. Camera motion reveals new surfaces, whereas intervention changes object motion, contact, and deformation. Current video world models are largely driven by appearance priors and often lose physical or spatial consistency over long horizons. We present TourPhysics, an online framework initialized from a single image and a declarative physical configuration. TourPhysics extends PhysOmni, our ACM Multimedia 2026 work, from finite physics-grounded video synthesis to persistent exploration and manipulation. TourPhysics combines deterministic simulation with video generation while assigning separate roles to simulator state, geometric evidence, generator controls, and appearance memory. For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated. Accepted observations publish the terminal state and update the appearance memory and subsequent generator controls, while the committed state and simulator geometry remain fixed throughout synthesis and retry. We further separate the simulator geometry used for projection and visibility from the relative depth used to condition the generator. A reference-anchored memory retrieves accepted static appearance through geometric cross-view correspondence and incorporates it through a bounded residual that reverts to the native path when no valid correspondence exists. On simulator-defined camera tours and object manipulations, TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines, preserves the input scene, and reduces appearance drift during long-horizon revisits.

Problem

• Physics-based
• World Models
• Consistency
• Exploration
• Manipulation

Innovation

• Persistent Exploration and Manipulation
• Deterministic Simulation
• Geometric Evidence
• Appearance Memory
• Reference-Anchored Memory