Landsat-Bench: Datasets and Benchmarks for Landsat Foundation Models

2025-06-10

AI Summary

The absence of standardized evaluation benchmarks for long-term Landsat imagery has hindered the development and fair comparison of foundational remote sensing models (GFMs). Method: To address this, we introduce Landsat-Bench—the first comprehensive benchmark suite specifically designed for Landsat data—covering classification, detection, and segmentation downstream tasks. We establish the first Landsat-specific evaluation framework, integrating SSL4EO-L self-supervised pretraining with multi-task transfer learning for unified model assessment. Results: Experiments demonstrate that SSL4EO-L significantly outperforms ImageNet-initialized baselines on Landsat-adapted tasks: +4.0% overall accuracy on EuroSAT-L and +5.1% mAP on BigEarthNet-L. This work establishes a new standard for evaluating Landsat-oriented foundational models and provides a reproducible, comparable evaluation infrastructure to advance large-scale remote sensing foundation models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The Landsat program offers over 50 years of globally consistent Earth imagery. However, the lack of benchmarks for this data constrains progress towards Landsat-based Geospatial Foundation Models (GFM). In this paper, we introduce Landsat-Bench, a suite of three benchmarks with Landsat imagery that adapt from existing remote sensing datasets -- EuroSAT-L, BigEarthNet-L, and LC100-L. We establish baseline and standardized evaluation methods across both common architectures and Landsat foundation models pretrained on the SSL4EO-L dataset. Notably, we provide evidence that SSL4EO-L pretrained GFMs extract better representations for downstream tasks in comparison to ImageNet, including performance gains of +4% OA and +5.1% mAP on EuroSAT-L and BigEarthNet-L.

Problem

• Lack of benchmarks for Landsat data limits GFM progress
• Introducing Landsat-Bench to standardize Landsat model evaluation
• SSL4EO-L pretrained models outperform ImageNet in downstream tasks

Innovation

• Introduces Landsat-Bench with three adapted benchmarks
• Uses SSL4EO-L pretrained models for better representations
• Shows +4% OA and +5.1% mAP performance gains