AlayaWorld: Long-Horizon and Playable Video World Generation

2026-07-07

AI Summary

Traditional game worlds rely on costly, hand-crafted pipelines that are difficult to modify and lack flexible, efficient generation mechanisms. This work proposes the first end-to-end open-source generative video world framework based on an autoregressive world model, which integrates gameplay footage and real-world video data to enable real-time generation of playable, open-ended virtual environments conditioned on current states and user interactions. The framework supports free navigation and diverse interactive actions such as combat and spellcasting, and unifies data preparation, model architecture, training, inference acceleration, and deployment within a modular design. It achieves long-horizon, high-interactivity simulation and is accompanied by a complete, reproducible toolchain, laying a foundation for embodied intelligence and interactive applications.

Abstract

Game worlds have traditionally been built through labor-intensive production pipelines, making them costly to develop, difficult to customization, and expensive to modify after deployment. Recent advances in video world models offer a fundamentally different paradigm. Rather than explicitly authoring every component of a virtual environment, these models autoregressively synthesize future observations conditioned on the current world state and user interactions, enabling playable worlds to be generated online. Trained on both gameplay recordings and real-world videos, they can capture diverse visual appearances and physical dynamics, opening new opportunities for interactive applications beyond gaming, including embodied intelligence. In this paper, we present \textbf{AlayaWorld}, a full-stack open-source framework for building interactive generative worlds. AlayaWorld enables open-ended real-time interaction, allowing users to freely navigate and perform diverse actions such as combat, spell casting, and monster summoning. The framework unifies the complete development-from data preparation model architecture, model training, inference acceleration, and deployment-within a modular and extensible architecture. Alongside the framework, we release reproducible pipelines, reference implementations, evaluation tools, and comprehensive documentation, establishing a practical foundation for future research and real-time applications of generative world models.

Problem

• video world generation
• interactive generative worlds
• long-horizon simulation
• playable environments
• embodied intelligence

Innovation

• generative world models
• interactive video generation
• real-time inference
• embodied intelligence
• open-source framework