World2Minecraft: Occupancy-Driven Simulated Scenes Construction

2026-04-30

AI Summary

Existing simulation platforms are hindered by data contamination and insufficient flexibility, limiting their utility for high-fidelity embodied intelligence research. To address this, this work proposes an automated pipeline based on 3D semantic occupancy prediction that enables low-cost, scalable reconstruction of real-world indoor scenes into structured Minecraft environments. The authors introduce MinecraftOcc, a large-scale dataset comprising 156 highly detailed scenes and 100,165 images, which substantially augments current resources. This dataset presents a significant challenge to state-of-the-art methods on downstream tasks such as vision-and-language navigation, thereby advancing the development of customizable simulation platforms for embodied artificial intelligence.

Abstract

Embodied intelligence requires high-fidelity simulation environments to support perception and decision-making, yet existing platforms often suffer from data contamination and limited flexibility. To mitigate this, we propose World2Minecraft to convert real-world scenes into structured Minecraft environments based on 3D semantic occupancy prediction. In the reconstructed scenes, we can effortlessly perform downstream tasks such as Vision-Language Navigation(VLN). However, we observe that reconstruction quality heavily depends on accurate occupancy prediction, which remains limited by data scarcity and poor generalization in existing models. We introduce a low-cost, automated, and scalable data acquisition pipeline for creating customized occupancy datasets, and demonstrate its effectiveness through MinecraftOcc, a large-scale dataset featuring 100,165 images from 156 richly detailed indoor scenes. Extensive experiments show that our dataset provides a critical complement to existing datasets and poses a significant challenge to current SOTA methods. These findings contribute to improving occupancy prediction and highlight the value of World2Minecraft in providing a customizable and editable platform for personalized embodied AI research. Project page:https://world2minecraft.github.io/.

Problem

• embodied intelligence
• 3D semantic occupancy prediction
• simulation environments
• data scarcity
• scene reconstruction

Innovation

• 3D semantic occupancy
• simulated environment construction
• embodied AI
• data acquisition pipeline
• Minecraft-based simulation