OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents

2026-02-19

AI Summary

This work addresses the limitations of existing remote sensing models in performing coherent multi-step reasoning across multi-scale spatial contexts, geographic structures, and multispectral indices. The authors propose a unified tool-augmented multimodal reasoning framework that trains a geospatial agent via supervised fine-tuning to conduct structured analysis by integrating satellite imagery with natural language queries. The core innovation lies in constructing the first large-scale geospatial multimodal training corpus annotated with explicit reasoning trajectories, coupled with integrated GIS operations and multispectral index computations (e.g., NDVI, NBR, NDBI). Evaluated across diverse tasks—including urban planning, environmental monitoring, disaster response, and infrastructure assessment—the model significantly outperforms strong baselines and achieves performance on par with state-of-the-art open- and closed-source systems.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent progress in multimodal reasoning has enabled agents that can interpret imagery, connect it with language, and perform structured analytical tasks. Extending such capabilities to the remote sensing domain remains challenging, as models must reason over spatial scale, geographic structures, and multispectral indices while maintaining coherent multi-step logic. To bridge this gap, OpenEarthAgent introduces a unified framework for developing tool-augmented geospatial agents trained on satellite imagery, natural-language queries, and detailed reasoning traces. The training pipeline relies on supervised fine-tuning over structured reasoning trajectories, aligning the model with verified multistep tool interactions across diverse analytical contexts. The accompanying corpus comprises 14,538 training and 1,169 evaluation instances, with more than 100K reasoning steps in the training split and over 7K reasoning steps in the evaluation split. It spans urban, environmental, disaster, and infrastructure domains, and incorporates GIS-based operations alongside index analyses such as NDVI, NBR, and NDBI. Grounded in explicit reasoning traces, the learned agent demonstrates structured reasoning, stable spatial understanding, and interpretable behaviour through tool-driven geospatial interactions across diverse conditions. We report consistent improvements over a strong baseline and competitive performance relative to recent open and closed-source models.

Problem

• geospatial reasoning
• remote sensing
• multimodal reasoning
• tool-augmented agents
• satellite imagery

Innovation

• tool-augmented agent
• geospatial reasoning
• multimodal remote sensing
• structured reasoning trajectories
• multispectral index analysis