Probabilistic Machine Learning for Noisy Labels in Earth Observation

2025-04-04

AI Summary

Label noise in Earth observation (EO) data severely undermines the robustness and reliability of supervised learning models. To address this, we propose the first input-dependent label noise modeling and uncertainty quantification framework tailored to EO scenarios. Our approach systematically integrates probabilistic machine learning—specifically Bayesian neural networks and deep probabilistic models—to jointly characterize the noise distribution induced by heterogeneous, multi-source remote sensing inputs. We further design a noise-aware loss function and establish an end-to-end pipeline for uncertainty calibration and evaluation. Extensive experiments across high-impact EO tasks—including land cover classification and cloud detection—demonstrate consistent and significant improvements over deterministic baselines. Crucially, our uncertainty estimates are rigorously validated: they exhibit strong calibration, interpretability, and cross-task generalizability—key properties for trustworthy EO decision support.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Remote Sensing / Geospatial AI
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Label noise poses a significant challenge in Earth Observation (EO), often degrading the performance and reliability of supervised Machine Learning (ML) models. Yet, given the critical nature of several EO applications, developing robust and trustworthy ML solutions is essential. In this study, we take a step in this direction by leveraging probabilistic ML to model input-dependent label noise and quantify data uncertainty in EO tasks, accounting for the unique noise sources inherent in the domain. We train uncertainty-aware probabilistic models across a broad range of high-impact EO applications-spanning diverse noise sources, input modalities, and ML configurations-and introduce a dedicated pipeline to assess their accuracy and reliability. Our experimental results show that the uncertainty-aware models consistently outperform the standard deterministic approaches across most datasets and evaluation metrics. Moreover, through rigorous uncertainty evaluation, we validate the reliability of the predicted uncertainty estimates, enhancing the interpretability of model predictions. Our findings emphasize the importance of modeling label noise and incorporating uncertainty quantification in EO, paving the way for more accurate, reliable, and trustworthy ML solutions in the field.

Problem

• Address label noise in Earth Observation ML models
• Develop robust probabilistic models for data uncertainty
• Enhance reliability of EO ML predictions

Innovation

• Leveraging probabilistic ML for label noise
• Training uncertainty-aware models across EO applications
• Introducing pipeline for accuracy and reliability assessment