Confidence-Filtered Relevance (CFR): An Interpretable and Uncertainty-Aware Machine Learning Framework for Naturalness Assessment in Satellite Imagery

2025-07-17

AI Summary

Current satellite imagery–based assessments of naturalness in protected areas suffer from limited interpretability and inadequate uncertainty quantification, hindering evaluation of model decision reliability. To address this, we propose the Certainty-aware Feature Relevance (CFR) framework—the first to integrate uncertainty-aware mechanisms into naturalness interpretability analysis. CFR jointly models feature relevance and epistemic uncertainty by unifying Layer-wise Relevance Propagation (LRP) with Deep Deterministic Uncertainty (DDU) estimation. Applied to the AnthroProtect dataset, CFR partitions samples via uncertainty thresholds and reveals that rising uncertainty significantly degrades attribution heatmap fidelity and increases entropy. Experiments demonstrate CFR’s robust identification of high-naturalness land covers—including shrubland, forest, and wetland—while quantifying uncertainty’s detrimental impact on explanation quality. By explicitly coupling interpretability with calibrated uncertainty, CFR substantially enhances transparency, trustworthiness, and robustness in remote sensing–based naturalness assessment.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

Protected natural areas play a vital role in ecological balance and ecosystem services. Monitoring these regions at scale using satellite imagery and machine learning is promising, but current methods often lack interpretability and uncertainty-awareness, and do not address how uncertainty affects naturalness assessment. In contrast, we propose Confidence-Filtered Relevance (CFR), a data-centric framework that combines LRP Attention Rollout with Deep Deterministic Uncertainty (DDU) estimation to analyze how model uncertainty influences the interpretability of relevance heatmaps. CFR partitions the dataset into subsets based on uncertainty thresholds, enabling systematic analysis of how uncertainty shapes the explanations of naturalness in satellite imagery. Applied to the AnthroProtect dataset, CFR assigned higher relevance to shrublands, forests, and wetlands, aligning with other research on naturalness assessment. Moreover, our analysis shows that as uncertainty increases, the interpretability of these relevance heatmaps declines and their entropy grows, indicating less selective and more ambiguous attributions. CFR provides a data-centric approach to assess the relevance of patterns to naturalness in satellite imagery based on their associated certainty.

Problem

• Lack interpretability in naturalness assessment methods
• Uncertainty-awareness missing in satellite imagery analysis
• Impact of uncertainty on relevance heatmaps interpretability

Innovation

• Combines LRP Attention Rollout with DDU estimation
• Partitions dataset based on uncertainty thresholds
• Assesses relevance of patterns with certainty