AutoUE: Automated Generation of 3D Games in Unreal Engine via Multi-Agent Systems

2026-03-07

AI Summary

This work addresses the challenges of automatically generating complete, executable 3D games within commercial game engines—namely, procedural complexity and high technical barriers—by proposing AutoUE, a multi-agent system that coordinates multiple specialized agents to end-to-end produce functional 3D games, encompassing scene construction, gameplay logic, and interactive code synthesis. The approach innovatively integrates retrieval-augmented generation to mitigate tool hallucinations in large language models, while incorporating constraints from Unreal Engine documentation and established game design patterns to ensure code correctness. An automated testing mechanism is further introduced to validate dynamic game behaviors. Evaluated on a newly curated dataset for game generation, experiments demonstrate AutoUE’s effectiveness in producing fully functional 3D games and confirm the overall system performance.

Technology Category

• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Multiagent Systems: Mechanism Design

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Automatically generating 3D games in commercial game engines remains a non-trivial challenge, as it involves complex engine-related workflows for generating assets such as scenes, blueprints, and code. To address this challenge, we propose a novel multi-agent system, AutoUE, which coordinates multiple agents to end-to-end generate 3D games, covering model retrieval, scene generation, gameplay and interaction code synthesis, and automated game testing for evaluation. In order to mitigate tool-use hallucinations in LLMs, we introduce a retrieval-augmented generation mechanism that grounds agents with relevant UE tool documentation. Additionally, we incorporate game design patterns and engine constraints into the code generation process to ensure the generation of correct and robust code. Furthermore, we design an automated play-testing pipeline that generates and executes runtime test commands, enabling systematic evaluation of dynamic behaviors. Finally, we construct a game generation dataset and conduct a series of experiments that demonstrate AutoUE's ability to generate 3D games end-to-end, and validate the effectiveness of these designs.

Problem

• 3D game generation
• Unreal Engine
• automated content creation
• multi-agent systems
• game development automation

Innovation

• multi-agent system
• retrieval-augmented generation
• automated game generation
• Unreal Engine
• code synthesis