Mind-Studio: Executable World Models with Lookahead Evaluation for Partially Observable Games

2026-06-14

AI Summary

Existing symbolic approaches struggle to generate environment models from interaction trajectories that are both independently executable and capable of capturing long-term dynamics, particularly in partially observable games. This work proposes a large language model–based method that automatically synthesizes Pygame-like executable world models from state–action–next-state trajectories, integrating game objects, scenes, and actions extracted from screenshots. To enhance completeness and generalization, the approach incorporates entropy-guided trajectory selection and lightweight skill files. Evaluated via a K-step lookahead fidelity protocol, the method dramatically improves next-state prediction accuracy on Montezuma’s Revenge—from 0.3% to 48.7%—successfully validates 5 out of 8 subgoals, and achieves superior branch-level fidelity compared to existing methods on Alien, Assault, and Skiing.

Abstract

World-model synthesis aims to turn interaction experience into an internal model of environment dynamics. Existing symbolic approaches often fit observed transitions or mixtures of local rules, but they do not produce a complete executable program that can run independently of the real environment. We present Mind-Studio, a framework that synthesizes executable pygame-style world models from state-action-next-state trajectories using large language models. Mind-Studio combines entropy-selected traces with a lightweight game skill file containing object, action, and static scene information extracted from screenshots. We evaluate synthesis quality with a K-step lookahead fidelity protocol that compares generated world-model rollouts against Real-ALE rollouts from the same state. On Montezuma's Revenge, Mind-Studio improves chosen-action next-state prediction from 0.3% for PoE-World to 48.7% while verifying 5 of 8 subgoals; across Alien, Assault, and Skiing, it achieves stronger branch-level fidelity than prior learned lookahead sources.

Problem

• world-model synthesis
• executable program
• partially observable games
• lookahead evaluation
• environment dynamics

Innovation

• executable world models
• lookahead evaluation
• large language models
• partially observable games
• world-model synthesis