LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models

2026-03-07

AI Summary

This work addresses the "out-of-view dynamics" problem in current generative video world models, which halt the state evolution of objects once they leave the field of view, thereby failing to reflect plausible dynamic changes upon their re-entry. To tackle this limitation, the authors propose LiveWorld, a novel framework that formally defines the problem and introduces a monitor-based mechanism to maintain a global scene state comprising a static 3D background and continuously evolving dynamic entities. By enabling autonomous temporal progression and state synchronization in unobserved regions—through persistent state representations, dynamic entity tracking, and spatially consistent rendering—LiveWorld facilitates authentic 4D dynamic world simulation. Evaluated on the newly introduced LiveBench benchmark, the method demonstrates significant improvements in long-term scene consistency and the coherent evolution of unobserved events.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control. However, they implicitly assume that the world only evolves within the observer's field of view. Once an object leaves the observer's view, its state is"frozen"in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime. In this work, we identify and formalize this overlooked limitation as the"out-of-sight dynamics"problem, which impedes video world models from representing a continuously evolving world. To address this issue, we propose LiveWorld, a novel framework that extends video world models to support persistent world evolution. Instead of treating the world as static observational memory, LiveWorld models a persistent global state composed of a static 3D background and dynamic entities that continue evolving even when unobserved. To maintain these unseen dynamics, LiveWorld introduces a monitor-based mechanism that autonomously simulates the temporal progression of active entities and synchronizes their evolved states upon revisiting, ensuring spatially coherent rendering. For evaluation, we further introduce LiveBench, a dedicated benchmark for the task of maintaining out-of-sight dynamics. Extensive experiments show that LiveWorld enables persistent event evolution and long-term scene consistency, bridging the gap between existing 2D observation-based memory and true 4D dynamic world simulation. The baseline and benchmark will be publicly available at https://zichengduan.github.io/LiveWorld/index.html.

Problem

• out-of-sight dynamics
• generative video world models
• persistent world evolution
• scene consistency
• dynamic entities

Innovation

• out-of-sight dynamics
• generative video world models
• persistent world evolution
• monitor-based simulation
• LiveWorld