Multi-Modal Contrastive Learning for Implicit Earth Embeddings via Location Tying

2026-06-18

AI Summary

This work addresses the scarcity of high-quality labeled data in geospatial prediction tasks by introducing multimodal contrastive learning into geospatial representation learning for the first time, proposing two novel architectures—MELT and SALT. Both models leverage a location-binding mechanism to enable self-supervised pretraining on unpaired, multisource data, thereby overcoming the limitations of conventional dual-modality alignment and supporting flexible multimodal fusion. The study identifies the location encoder as a key performance bottleneck and demonstrates that both models achieve performance on par with SATCLIP—the current state-of-the-art dual-modality baseline—across four downstream tasks. Notably, MELT exhibits more stable training dynamics, establishing a robust foundation for future extensions to richer multimodal settings.

Abstract

Spatial prediction tasks are often limited by a lack of high-quality labelled ground-truth observations. To overcome this challenge, self-supervised pre-training is a possible solution, with contrastive learning dominant for location encoders. Those approaches usually align geographic coordinates with just one additional modality. We propose two multimodal contrastive learning architectures: Multimodal Embedding via Location Tying (MELT) and Sequential Alternating Location Training (SALT). These architectures expand this framework beyond two modalities by utilising unpaired geospatial data. Both methods are technically viable and match the performance of the strongest two-modality baseline (SATCLIP) across four downstream tasks. However, increasing the number of modalities does not consistently improve performance, suggesting that the chosen location encoder is the main limitation - the contrastive objective reaches its peak early, regardless of modality diversity or pre-training volume. MELT provides more stable training than SALT and presents a stronger foundation for future scaling.

Problem

• spatial prediction
• self-supervised pre-training
• contrastive learning
• multi-modal learning
• geospatial data

Innovation

• multimodal contrastive learning
• location tying
• self-supervised pre-training
• geospatial embedding
• unpaired data