WorldGen: From Text to Traversable and Interactive 3D Worlds

2025-11-20

AI Summary

This work addresses the challenge of generating large-scale, semantically coherent, geometrically consistent, and style-controllable interactive 3D worlds from text alone—without manual modeling or specialized 3D expertise. Methodologically, we propose a modular generative framework: (1) a large language model for scene-level semantic understanding and layout planning; (2) procedural generation combined with object-aware scene decomposition for structural coherence; and (3) diffusion-based 3D synthesis to ensure high-fidelity, geometry-aware content generation. The resulting scenes are exportable to mainstream game engines, enabling real-time rendering, interactive editing, and navigation. Experiments demonstrate substantial improvements in both generation efficiency and visual quality: our approach supports平方公里-scale, semantically rich, interactive 3D world construction while maintaining high frame rates. This establishes a novel paradigm for AI-generated content (AIGC)-driven virtual environment creation.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We introduce WorldGen, a system that enables the automatic creation of large-scale, interactive 3D worlds directly from text prompts. Our approach transforms natural language descriptions into traversable, fully textured environments that can be immediately explored or edited within standard game engines. By combining LLM-driven scene layout reasoning, procedural generation, diffusion-based 3D generation, and object-aware scene decomposition, WorldGen bridges the gap between creative intent and functional virtual spaces, allowing creators to design coherent, navigable worlds without manual modeling or specialized 3D expertise. The system is fully modular and supports fine-grained control over layout, scale, and style, producing worlds that are geometrically consistent, visually rich, and efficient to render in real time. This work represents a step towards accessible, generative world-building at scale, advancing the frontier of 3D generative AI for applications in gaming, simulation, and immersive social environments.

Problem

• Automatically creating large-scale interactive 3D worlds from text prompts
• Transforming natural language descriptions into traversable textured environments
• Bridging creative intent and functional virtual spaces without manual modeling

Innovation

• Text-driven automatic creation of interactive 3D worlds
• Combines LLM reasoning, procedural and diffusion-based 3D generation
• Produces modular, traversable environments for real-time rendering