TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data

2025-04-15

AI Summary

Remote sensing foundation models are hindered by small-scale, geographically narrow, and single-modality training datasets, limiting label-efficient large-scale pretraining. To address this, we introduce GeoEarth—the first global-scale, multimodal, spatiotemporally aligned Earth observation dataset—integrating eight modalities: optical, SAR, digital elevation, land cover, and others, spanning over 9 million globally distributed samples. GeoEarth is the first to systematically achieve co-registration, standardization, and spatiotemporal alignment of Analysis-Ready Data (ARD) across all eight modalities at planetary scale, thereby overcoming critical bottlenecks in modality diversity, geographic coverage, and data readiness. Extensive experiments demonstrate substantial performance gains on downstream tasks—including land-cover classification and change detection. The dataset is released with comprehensive metadata, detailed processing documentation, benchmark pretraining protocols, and a permissive open-source license.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Large-scale foundation models in Earth Observation can learn versatile, label-efficient representations by leveraging massive amounts of unlabeled data. However, existing public datasets are often limited in scale, geographic coverage, or sensor variety. We introduce TerraMesh, a new globally diverse, multimodal dataset combining optical, synthetic aperture radar, elevation, and land-cover modalities in an Analysis-Ready Data format. TerraMesh includes over 9 million samples with eight spatiotemporal aligned modalities, enabling large-scale pre-training and fostering robust cross-modal correlation learning. We provide detailed data processing steps, comprehensive statistics, and empirical evidence demonstrating improved model performance when pre-trained on TerraMesh. The dataset will be made publicly available with a permissive license.

Problem

• Lack of large-scale diverse Earth Observation datasets
• Limited geographic coverage and sensor variety in existing datasets
• Need for robust cross-modal correlation learning in foundation models

Innovation

• Globally diverse multimodal dataset TerraMesh
• Eight spatiotemporal aligned modalities included
• Analysis-Ready Data format for pre-training