IC-EO: Interpretable Code-based assistant for Earth Observation

2026-01-27

AI Summary

This work addresses the high barrier to entry and lack of auditability and reproducibility in Earth observation (EO) analysis, which often relies on opaque, black-box systems. To overcome these limitations, the authors propose the first three-tiered, controllable, and transparent EO analysis framework—comprising tool, agent, and task layers—that leverages tool-augmented large language models to automatically translate natural language queries into executable and verifiable Python workflows. The framework supports a range of EO tasks, including classification, segmentation, detection, spectral index computation, and geospatial operations. Evaluated on land cover mapping and post-wildfire damage assessment, the system achieves accuracies of 64.2% and 50%, respectively, significantly outperforming general-purpose baselines such as GPT-4o and LLaVA, while ensuring interpretability and reproducibility of results.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Evaluation and Analysis
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Despite recent advances in computer vision, Earth Observation (EO) analysis remains difficult to perform for the laymen, requiring expert knowledge and technical capabilities. Furthermore, many systems return black-box predictions that are difficult to audit or reproduce. Leveraging recent advances in tool LLMs, this study proposes a conversational, code-generating agent that transforms natural-language queries into executable, auditable Python workflows. The agent operates over a unified easily extendable API for classification, segmentation, detection (oriented bounding boxes), spectral indices, and geospatial operators. With our proposed framework, it is possible to control the results at three levels: (i) tool-level performance on public EO benchmarks; (ii) at the agent-level to understand the capacity to generate valid, hallucination-free code; and (iii) at the task-level on specific use cases. In this work, we select two use-cases of interest: land-composition mapping and post-wildfire damage assessment. The proposed agent outperforms general-purpose LLM/VLM baselines (GPT-4o, LLaVA), achieving 64.2% vs. 51.7% accuracy on land-composition and 50% vs. 0% on post-wildfire analysis, while producing results that are transparent and easy to interpret. By outputting verifiable code, the approach turns EO analysis into a transparent, reproducible process.

Problem

• Earth Observation
• black-box predictions
• expert knowledge
• reproducibility
• interpretability

Innovation

• interpretable AI
• code-generating agent
• Earth Observation
• tool-augmented LLM
• reproducible analysis