How To Embed Matters: Evaluation of EO Embedding Design Choices

2026-03-11

AI Summary

This study addresses the design of efficient, general-purpose, and scalable embedding representations for Earth observation (EO) tasks by systematically evaluating key architectural and training choices when using GeoFM as a feature extractor. These include backbone architecture, pretraining strategy, representation depth, spatial aggregation, and multi-objective fusion. Experiments on the NeuCo-Bench benchmark demonstrate that a Transformer backbone combined with mean pooling establishes a strong baseline; intermediate ResNet layers outperform final-layer features; self-supervised objectives offer task-specific advantages; and multi-objective fusion substantially enhances robustness. The resulting embeddings compress the original input by over 500× while maintaining high performance across diverse downstream EO tasks, underscoring the critical role of thoughtful embedding design in enabling scalable EO workflows.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Feature Construction/Reformulation
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Earth observation (EO) missions produce petabytes of multispectral imagery, increasingly analyzed using large Geospatial Foundation Models (GeoFMs). Alongside end-to-end adaptation, workflows make growing use of intermediate representations as task-agnostic embeddings, enabling models to compute representations once and reuse them across downstream tasks. Consequently, when GeoFMs act as feature extractors, decisions about how representations are obtained, aggregated, and combined affect downstream performance and pipeline scalability. Understanding these trade-offs is essential for scalable embedding-based EO workflows, where compact embeddings can replace raw data while remaining broadly useful. We present a systematic analysis of embedding design in GeoFM-based EO workflows. Leveraging NeuCo-Bench, we study how backbone architecture, pretraining strategy, representation depth, spatial aggregation, and representation combination influence EO task performance. We demonstrate the usability of GeoFM embeddings by aggregating them into fixed-size representations more than 500x smaller than the raw input data. Across models, we find consistent trends: transformer backbones with mean pooling provide strong default embeddings, intermediate ResNet layers can outperform final layers, self-supervised objectives exhibit task-specific strengths, and combining embeddings from different objectives often improves robustness.

Problem

• Earth Observation
• Geospatial Foundation Models
• Embedding Design
• Feature Extraction
• Scalable Workflows

Innovation

• GeoFM embeddings
• embedding design
• spatial aggregation
• self-supervised pretraining
• multispectral compression