Craftium: An Extensible Framework for Creating Reinforcement Learning Environments

2024-07-04

arXiv.org

AI Summary

Existing scalable 2D environments inadequately model real-world challenges such as 3D navigation and spatial reasoning, while mainstream 3D environments suffer from high computational overhead, poor customizability, and limited native multi-agent support. To address these limitations, we propose VoxRL—a novel, efficient 3D visual reinforcement learning framework that deeply integrates the modular voxel engine Minetest with the standard RL interface Gymnasium. VoxRL enables procedurally generated infinite worlds, flexible customization via Lua/Python scripting, native multi-agent interaction, and lightweight rendering. We open-source five benchmark environments, along with full code and comprehensive documentation, substantially lowering the barrier to constructing high-fidelity 3D RL environments. Empirical evaluations demonstrate VoxRL’s effectiveness in supporting algorithmic studies on spatial representation learning and multi-agent coordination.

Technology Category

• Machine Learning: Reinforcement Learning
• Computer Vision: 3D Computer Vision
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Web Mining and Content Analysis: Web data generation and simulation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Most Reinforcement Learning (RL) environments are created by adapting existing physics simulators or video games. However, they usually lack the flexibility required for analyzing specific characteristics of RL methods often relevant to research. This paper presents Craftium, a novel framework for exploring and creating rich 3D visual RL environments that builds upon the Minetest game engine and the popular Gymnasium API. Minetest is built to be extended and can be used to easily create voxel-based 3D environments (often similar to Minecraft), while Gymnasium offers a simple and common interface for RL research. Craftium provides a platform that allows practitioners to create fully customized environments to suit their specific research requirements, ranging from simple visual tasks to infinite and procedurally generated worlds. We also provide five ready-to-use environments for benchmarking and as examples of how to develop new ones. The code and documentation are available at https://github.com/mikelma/craftium/.

Problem

• Lack of flexible tools for rich 3D agent environments
• High computational cost of complex 3D environments
• Missing customizability and multi-agent support in existing platforms

Innovation

• Customizable 3D single- and multi-agent platform
• Efficient procedural task generators
• High performance with +2K steps per second