🤖 AI Summary
This work addresses two key challenges in game development: insufficient integration between AI coding assistants and design workflows, and limited modalities in human–AI interaction. To this end, we propose DreamGarden—a large language model (LLM)-driven game design assistant tailored for Unreal Engine. Our method processes a single-sentence natural-language prompt (e.g., a dream or scene description) through a hierarchical LLM planner to generate an executable, multi-level design plan; modular subsystems then orchestrate engine-specific APIs to enable end-to-end content generation. Innovatively, we introduce the “game growth” metaphor and a visual “garden” interface supporting interactive seeding, pruning, and real-time feedback loops. Our contributions include: (1) the first planning-execution decoupled architecture for game design, establishing an open-ended human–AI co-creation paradigm; and (2) empirical validation showing that DreamGarden significantly lowers the entry barrier for non-expert users and enables rapid prototyping and iterative development of diverse, open-ended game environments.
📝 Abstract
Coding assistants are increasingly leveraged in game design, both generating code and making high-level plans. To what degree can these tools align with developer workflows, and what new modes of human-computer interaction can emerge from their use? We present DreamGarden, an AI system capable of assisting with the development of diverse game environments in Unreal Engine. At the core of our method is an LLM-driven planner, capable of breaking down a single, high-level prompt—a dream, memory, or imagined scenario provided by a human user—into a hierarchical action plan, which is then distributed across specialized submodules facilitating concrete implementation. This system is presented to the user as a garden of plans and actions, both growing independently and responding to user intervention via seed prompts, pruning, and feedback. Through a user study, we explore design implications of this system, charting courses for future work in semi-autonomous assistants and open-ended simulation design.