EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia

2025-04-28

AI Summary

This paper addresses the challenge of predicting ecological attributes (e.g., land cover, climate, soil) from satellite imagery under weak supervision. We propose a remote sensing vision-language modeling paradigm that jointly leverages species occurrence locations and Wikipedia-based habitat descriptions. Our key contributions are threefold: (1) We introduce EcoWikiRS, the first multi-source aligned dataset integrating high-resolution remote sensing imagery, GBIF species occurrence records, and Wikipedia habitat textual descriptions; (2) We design Weighted InfoNCE Loss (WINCEL), which explicitly mitigates noise and local bias arising from weak text–image alignment; (3) We achieve zero-shot ecological semantic understanding, significantly outperforming prior methods on the EUNIS ecosystem classification benchmark. All code and data are publicly released.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Semi-Supervised Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The presence of species provides key insights into the ecological properties of a location such as land cover, climatic conditions or even soil properties. We propose a method to predict such ecological properties directly from remote sensing (RS) images by aligning them with species habitat descriptions. We introduce the EcoWikiRS dataset, consisting of high-resolution aerial images, the corresponding geolocated species observations, and, for each species, the textual descriptions of their habitat from Wikipedia. EcoWikiRS offers a scalable way of supervision for RS vision language models (RS-VLMs) for ecology. This is a setting with weak and noisy supervision, where, for instance, some text may describe properties that are specific only to part of the species' niche or is irrelevant to a specific image. We tackle this by proposing WINCEL, a weighted version of the InfoNCE loss. We evaluate our model on the task of ecosystem zero-shot classification by following the habitat definitions from the European Nature Information System (EUNIS). Our results show that our approach helps in understanding RS images in a more ecologically meaningful manner. The code and the dataset are available at https://github.com/eceo-epfl/EcoWikiRS.

Problem

• Predict ecological properties from satellite images using species data
• Align remote sensing images with species habitat descriptions
• Improve ecological understanding of satellite images via weak supervision

Innovation

• Aligns satellite images with species habitat descriptions
• Uses weak supervision from species data and Wikipedia
• Introduces WINCEL loss for noisy ecological supervision