UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation

2026-06-22

AI Summary

This work addresses the challenge of limited generalizability of Vision Transformers (ViTs) in Earth observation due to disparities in sensor modalities, spatial scales, and resolutions. To overcome this, the authors propose UniverSat, a ViT architecture built upon a universal patch encoder that maps heterogeneous remote sensing data—spanning arbitrary spatial, spectral, and temporal resolutions, as well as optical and non-optical modalities—into a unified embedding space through shared-weight projections. Coupled with self-supervised pretraining, UniverSat enables unified representation learning across diverse multimodal inputs without relying on fixed patch projections, a constraint inherent in conventional ViTs. The model achieves state-of-the-art performance on multiple benchmarks, including GeoBench, PANGEABench, and SpectralEarth, consistently outperforming existing methods in both classification and segmentation tasks.

Abstract

Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where input modalities, scales, and resolutions vary widely. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a shared set of weights. This enables training a single model on heterogeneous multimodal corpora via self-supervision, yielding robust, sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. Our code and models are available at https://github.com/gastruc/UniverSat.

Problem

• Earth Observation
• Vision Transformers
• multimodal
• resolution-agnostic
• modality-agnostic

Innovation

• Universal Patch Encoder
• Resolution-Agnostic
• Modality-Agnostic
• Self-Supervised Learning
• Earth Observation