Probing Geospatial SSL Representations with Environmental Signals

2026-07-06

AI Summary

This study addresses the limitations of current evaluation practices for geospatial self-supervised learning (SSL) representations, which predominantly rely on downstream tasks and fail to reveal the encoded environmental information. To overcome this, the work introduces co-located ERA5 reanalysis environmental variables—such as temperature and precipitation—as physically grounded probes aligned with satellite imagery. By integrating linear probing with representational geometry metrics, the authors systematically assess the capacity of representations learned by models like DINO, MAE, and MoCo to retain environmental signals. The findings demonstrate that geometric properties of representation layers effectively differentiate models with similar downstream performance, and that the linear accessibility of environmental signals strongly correlates with performance on environment-dependent tasks in PANGAEA. The project also releases an ERA5-annotated dataset aligned with SSL4EO, offering a novel perspective for representation evaluation beyond task-driven paradigms.

Abstract

Self-supervised learning (SSL) is designed to learn generic, transferable representations rather than representations optimized for a single task. Most geospatial benchmarks evaluate representations solely through downstream tasks, providing limited insight into the information encoded within the representation itself. We ask a different question: do SSL representations of satellite imagery preserve statistical associations with environmental variables that co-vary with the imaging process? To answer this question, we probe SSL representations using co-located ERA5 reanalysis variables, a global dataset of physically consistent environmental variables, including temperature, precipitation, surface solar radiation, surface pressure, and volumetric soil water. These variables are physically related to the spectral reflectance and radar backscatter recorded by Sentinel-1 and Sentinel-2, making them meaningful evaluation targets despite not being used during SSL pretraining. We complement this probing analysis with intrinsic representation metrics to characterize representation geometry and investigate how these properties relate to downstream performance and the encoding of environmental signals. Using DINO, MAE, and MoCo models trained under identical conditions, we show that representation-level metrics distinguish models with similar downstream benchmark performance, providing complementary information beyond task-driven benchmarks. We further find that the linear accessibility of environmental signals is associated with performance on environmentally dependent tasks in the PANGAEA benchmark. Finally, we release ERA5 annotations co-located with the SSL4EO dataset to enable physically grounded representation evaluation for future geospatial foundation models.

Problem

• self-supervised learning
• geospatial representation
• environmental signals
• representation probing
• satellite imagery

Innovation

• self-supervised learning
• geospatial representation
• environmental probing
• representation geometry
• ERA5 reanalysis