MESA: Text-Driven Terrain Generation Using Latent Diffusion and Global Copernicus Data

2025-04-09

AI Summary

To address the limitations of conventional procedural terrain modeling—namely, heavy reliance on hand-crafted expert rules and poor generalizability—this paper introduces the first text-driven, high-fidelity terrain generation framework. Methodologically, we build upon a Latent Diffusion Model (LDM), integrating a CLIP text encoder with a digital elevation model (DEM)-conditioned control module; crucially, we construct the first large-scale remote sensing–text paired training dataset using global Copernicus DEM data and design a dedicated preprocessing pipeline to ensure geographic consistency. Our contributions are threefold: (1) establishing an open, scalable, data-driven paradigm for terrain synthesis; (2) publicly releasing the Major TOM Core-DEM dataset; and (3) generating diverse, geometrically plausible, and semantically aligned terrain samples that significantly outperform existing methods in both realism and cross-scenario generalization.

Technology Category

• Natural Language Processing: Generation
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Terrain modeling has traditionally relied on procedural techniques, which often require extensive domain expertise and handcrafted rules. In this paper, we present MESA - a novel data-centric alternative by training a diffusion model on global remote sensing data. This approach leverages large-scale geospatial information to generate high-quality terrain samples from text descriptions, showcasing a flexible and scalable solution for terrain generation. The model's capabilities are demonstrated through extensive experiments, highlighting its ability to generate realistic and diverse terrain landscapes. The dataset produced to support this work, the Major TOM Core-DEM extension dataset, is released openly as a comprehensive resource for global terrain data. The results suggest that data-driven models, trained on remote sensing data, can provide a powerful tool for realistic terrain modeling and generation.

Problem

• Generates terrain from text using diffusion models
• Replaces procedural methods with data-driven approach
• Leverages global remote sensing data for realism

Innovation

• Uses latent diffusion for terrain generation
• Leverages global Copernicus remote sensing data
• Generates terrain from text descriptions