TESSERA v2: Scaling Pixel-wise Earth Foundation Models

2026-07-04

AI Summary

This work addresses the inefficiency of computational budget allocation in pixel-level pretraining of foundation models for Earth observation and the weak correlation between pretraining loss and downstream performance. Through 395 controlled scaling experiments spanning 15 downstream tasks, the study proposes a novel compute allocation strategy that synchronously scales the encoder and training data while keeping the projection head fixed, and selects models based on downstream performance rather than pretraining loss. Integrating the Barlow Twins self-supervised framework, thousand-GPU GH200 distributed training, knowledge distillation, and Matryoshka embeddings, the authors distill an efficient model, TESSERA v2-1B-M (21 million parameters), which surpasses both open- and closed-source counterparts in overall performance. Its 16-dimensional embedding retains 92% of full-model performance while reducing storage to one-eighth, and the team will release a global embedding dataset covering 2017–2025.

Abstract

Pixel-wise Earth-observation (EO) foundation models are now achieving state-of-the-art performance via generated spatial embeddings. However, how these models scale and how best to spend a pretraining budget remain poorly understood. We present the largest controlled scaling study for EO to date: 395 training runs on 1,024 GH200 superchips within a fixed pixel-wise Barlow Twins family, each evaluated on 15 downstream tasks. We find that pretraining loss barely predicts downstream performance (|Pearson r| < 0.2), so selecting models by loss wastes a large share of the compute. We also find that, as the training budget grows, the encoder and the data should grow together while the projector stays fixed, which gives a simple rule for allocating compute. Using this rule, we train a family of pixel-wise models (0.5B and 1B, with a 2B model in training) and distill them into compact students for embeddings-as-data deployment. The 21-million-parameter distilled TESSERA v2-1B-M in aggregate outperforms all open and proprietary models tested, some of which are orders of magnitude larger. These students produce Matryoshka representations that are inexpensive to serve: a 16-dimensional prefix keeps 92% of the full 128-dimensional performance at 1/8 of the storage. Upon completion of training we plan to release v2 global embeddings covering 2017-2025. Together, these results give a concrete, empirically grounded recipe for scaling pixel-wise EO foundation models: train large encoders, select by downstream performance, and distil into flexible student models. All code will be released at https://github.com/ucam-eo/tessera.

Problem

• scaling
• Earth observation
• foundation models
• pretraining budget
• pixel-wise

Innovation

• scaling laws
• pixel-wise foundation models
• knowledge distillation
• Matryoshka representations
• Earth observation