IOI: Decoupling Kinematics and Physics for Interactive World Models

2026-06-22

AI Summary

This work addresses the geometric inconsistency and physical implausibility often exhibited by existing data-driven interactive world models due to the absence of explicit structural constraints, which leads to spatiotemporal drift between actions and visual feedback. To overcome this, the paper introduces the IOI hybrid world model, which explicitly incorporates analytical kinematic priors decoupled from learned physical dynamics. The model employs forward kinematics to generate deterministic trajectories and leverages multi-view orthogonal projection rendering to enforce geometric consistency. A novel multi-view kinematic aggregation and injection module enables calibration-free fusion across views. Evaluated on the RoboTwin benchmark, the method achieves state-of-the-art performance, significantly enhancing motion fidelity and out-of-distribution generalization. Policies trained on its synthetic data rival teleoperated demonstrations, and the model serves as a policy evaluator highly consistent with real-world physics simulation.

Abstract

Developing generalist embodied agents requires interactive environments providing visually realistic feedback and accurate action-conditioned dynamics. Interactive world models address this by simulating such complex dynamics. However, purely data-driven methods struggle to ensure precise control alignment and physically plausible visual feedback due to a lack of explicit structural constraints. To address this, we propose IOI, a hybrid interactive world model integrating analytical kinematic priors with learned physical dynamics. Unlike data-driven approaches prone to spatiotemporal drift, IOI introduces explicit kinematic guidance, computing forward kinematics from action sequences for accurate motion trajectories. These trajectories are rendered into synchronized front, side, and top orthographic projections, eliminating the need for extrinsic camera calibration. A Multi-view Kinematic Aggregation and Injection module fuses these geometric cues and injects them into the video generator, providing geometry-consistent guidance. Conditioning video generation on these deterministic trajectories establishes a synergy between the analytical simulator and the world model. Decoupling deterministic motion into the kinematic prior frees the generator to model stochastic physical interactions. Experiments on the RoboTwin benchmark validate IOI across kinematic fidelity, out-of-distribution (OOD) generalization, and policy evaluation. IOI achieves state-of-the-art simulation performance and robust zero-shot generalization to unseen OOD tasks. Furthermore, IOI serves as a reliable policy evaluator, yielding success rates closely aligning with ground-truth physics simulators. On real-world platforms, policies trained on IOI-synthesized data match those trained on teleoperation demonstrations, solidifying its practical value for embodied policy learning.

Problem

• interactive world models
• kinematics
• physical dynamics
• control alignment
• visual feedback

Innovation

• interactive world models
• kinematic-physical decoupling
• forward kinematics guidance
• multi-view orthographic rendering
• policy evaluation