Einstein World Models

2026-06-25

AI Summary

This work investigates whether intelligent reasoning requires counterfactual simulation that transcends direct experience, particularly in complex scenarios where linguistic expression is insufficient, by introducing visual thought experiments. To this end, we propose a novel reasoning framework that extends the tool-use capabilities of large language models into the visual-temporal domain for the first time, enabling them to actively invoke external world modules to generate testable counterfactual visual trajectories and integrate multimodal information for reasoning. Experimental results demonstrate that this approach significantly enhances model performance on spatial, physical, and causal reasoning tasks, thereby validating the efficacy of visualized hypothetical simulation in supporting complex cognitive processes.

Abstract

Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought. We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities. Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning. Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments.

Problem

• counterfactual reasoning
• visualisation
• large language models
• reasoning
• world models

Innovation

• Einstein World Models
• visual-temporal rollouts
• counterfactual reasoning
• world-module
• tool-augmented LLMs