Uncertainty evaluation of segmentation models for Earth observation

2025-10-22

AI Summary

Current remote sensing semantic segmentation lacks systematic evaluation of pixel-wise uncertainty estimation. This paper presents the first benchmark study of mainstream uncertainty quantification methods—including stochastic segmentation networks, model ensembles, and confidence metrics across diverse neural architectures—within Earth observation. Using the PASTIS and ForTy remote sensing datasets, we systematically evaluate error detection performance and noise sensitivity across varying geographical scales and label quality. Results reveal domain-specific generalization biases in several methods, while specific ensemble strategies and calibration-aware confidence metrics substantially improve uncertainty reliability. We propose a set of remote sensing–oriented uncertainty evaluation principles and practical guidelines, addressing a critical methodological gap in uncertainty validation. The findings provide deployable technical foundations for high-assurance automated land-cover interpretation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Remote Sensing / Geospatial AI

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

This paper investigates methods for estimating uncertainty in semantic segmentation predictions derived from satellite imagery. Estimating uncertainty for segmentation presents unique challenges compared to standard image classification, requiring scalable methods producing per-pixel estimates. While most research on this topic has focused on scene understanding or medical imaging, this work benchmarks existing methods specifically for remote sensing and Earth observation applications. Our evaluation focuses on the practical utility of uncertainty measures, testing their ability to identify prediction errors and noise-corrupted input image regions. Experiments are conducted on two remote sensing datasets, PASTIS and ForTy, selected for their differences in scale, geographic coverage, and label confidence. We perform an extensive evaluation featuring several models, such as Stochastic Segmentation Networks and ensembles, in combination with a number of neural architectures and uncertainty metrics. We make a number of practical recommendations based on our findings.

Problem

• Estimating uncertainty in satellite image segmentation predictions
• Benchmarking uncertainty methods for remote sensing applications
• Evaluating uncertainty measures for identifying prediction errors

Innovation

• Benchmarks uncertainty methods for satellite imagery
• Evaluates uncertainty for pixel-level segmentation errors
• Tests methods on remote sensing datasets PASTIS and ForTy