Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Models

2026-06-23

AI Summary

Existing video generation models lack support for action intervention, physical constraints, and counterfactual reasoning, hindering the development of controllable and evolvable world models. This work proposes a novel autonomous video generation framework that, for the first time, treats counterfactual controllability as a core principle. By integrating an action intervention mechanism and a physics-aware constraint verification module, the model generates physically plausible futures that respond coherently to interventions. Furthermore, it leverages generative feedback to enable continuous self-evolution. This approach transcends conventional purely visual prediction paradigms, substantially enhancing controllability, generalization, and autonomous evolutionary capacity in complex dynamic environments.

Abstract

Existing literature claims that video generation essentially is world modelling. On the one hand, the claim is productive because it pushes generative AI beyond static images and toward temporally extended physical scenes. On the other hand, this claim dangerously relies on the belief that scaling visual prediction alone will automatically yield physical agents. We prefer a more accurate statement: video generation models learn a partial, implicit spatiotemporal world model, but not a fully grounded or controllable one. The reason is as follows: a model may generate a plausible video of a drone crossing a forest or a robot arm manipulating a cup, yet still fail to know which variables are controllable, which constraints belong to a particular body and which futures remain valid under intervention. The frontier in essence is not predictive realism alone, instead it emphasizes a self-evolving generative nature that requires the decisive criterion to be counterfactual controllability: the capability of asking what would happen under an action, to test whether the generated future can survive embodiment constraints and to feed the resulting action knowledge back into future imagination (generation). Therefore, in this paper we present a new perspective, i.e., autonomous video generation with counterfactual controllability is one promising way to realize self-evolving world models.

Problem

• video generation
• world models
• counterfactual controllability
• self-evolving
• autonomous generation

Innovation

• counterfactual controllability
• autonomous video generation
• self-evolving world models
• embodied AI
• temporal world modeling