Unreal-MAP: Unreal-Engine-Based General Platform for Multi-Agent Reinforcement Learning

2025-03-20

AI Summary

To address the lack of high-fidelity, customizable, and algorithm-friendly simulation platforms in multi-agent reinforcement learning (MARL) research, this paper introduces the first general-purpose MARL platform deeply integrated with Unreal Engine 5 (UE5). The platform enables seamless integration of UE5’s physics and visual resources into MARL training loops via a hybrid Python/C++ interface compatible with ROS and ML-Agents. Its modular, open-source architecture—coupled with a cross-algorithm experiment management toolchain—supports flexible construction of 3D multi-agent tasks and rapid deployment of state-of-the-art algorithms including PPO, MAPPO, and QMix. Experiments demonstrate substantial improvements in environmental fidelity and task scalability, while providing unified visualization, analytical capabilities, and standardized benchmarking. This platform bridges the gap between MARL research and complex real-world applications, accelerating the transition of MARL toward deployment in realistic, dynamic environments.

Technology Category

• Multiagent Systems: Multiagent Planning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

In this paper, we propose Unreal Multi-Agent Playground (Unreal-MAP), an MARL general platform based on the Unreal-Engine (UE). Unreal-MAP allows users to freely create multi-agent tasks using the vast visual and physical resources available in the UE community, and deploy state-of-the-art (SOTA) MARL algorithms within them. Unreal-MAP is user-friendly in terms of deployment, modification, and visualization, and all its components are open-source. We also develop an experimental framework compatible with algorithms ranging from rule-based to learning-based provided by third-party frameworks. Lastly, we deploy several SOTA algorithms in example tasks developed via Unreal-MAP, and conduct corresponding experimental analyses. We believe Unreal-MAP can play an important role in the MARL field by closely integrating existing algorithms with user-customized tasks, thus advancing the field of MARL.

Problem

• Develops Unreal-MAP for multi-agent reinforcement learning tasks.
• Integrates Unreal Engine resources with MARL algorithms.
• Facilitates user-friendly deployment and visualization of MARL experiments.

Innovation

• Unreal-Engine-based multi-agent reinforcement learning platform
• Open-source, user-friendly task creation and algorithm deployment
• Compatible with rule-based and learning-based algorithms