Narrative-to-Scene Generation: An LLM-Driven Pipeline for 2D Game Environments

2025-08-30

AI Summary

This work addresses the long-standing challenge in procedural content generation (PCG) of automatically transforming narrative text into playable visual environments. We propose a lightweight generative pipeline that parses natural-language stories into spatial predicate triplets (Object–Relation–Object), uniquely integrating large language models’ (LLMs) narrative comprehension with GameTileNet’s affordance-aware semantic embeddings to drive semantically grounded 2D tile-based scene layout. Terrain generation across multiple frames is achieved via cellular automata, while object placement adheres to spatial constraints encoded as logical rules. Evaluated on ten diverse narratives, our approach significantly improves semantic object–environment alignment, terrain layering coherence, and cross-frame spatial consistency. Results demonstrate the feasibility and scalability of generating playable, narrative-driven scenes grounded in semantically meaningful, action-oriented representations.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Natural Language Processing: Generation
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Recent advances in large language models(LLMs) enable compelling story generation, but connecting narrative text to playable visual environments remains an open challenge in procedural content generation(PCG). We present a lightweight pipeline that transforms short narrative prompts into a sequence of 2D tile-based game scenes, reflecting the temporal structure of stories. Given an LLM-generated narrative, our system identifies three key time frames, extracts spatial predicates in the form of "Object-Relation-Object" triples, and retrieves visual assets using affordance-aware semantic embeddings from the GameTileNet dataset. A layered terrain is generated using Cellular Automata, and objects are placed using spatial rules grounded in the predicate structure. We evaluated our system in ten diverse stories, analyzing tile-object matching, affordance-layer alignment, and spatial constraint satisfaction across frames. This prototype offers a scalable approach to narrative-driven scene generation and lays the foundation for future work on multi-frame continuity, symbolic tracking, and multi-agent coordination in story-centered PCG.

Problem

• Connecting narrative text to playable visual environments
• Transforming narrative prompts into 2D game scenes
• Generating scenes reflecting temporal story structure

Innovation

• LLM-driven narrative-to-scene generation pipeline
• Affordance-aware semantic embeddings for asset retrieval
• Cellular Automata terrain with predicate-based object placement