Uncertainty-Aware Cross-Modal Remote Sensing Image-Text Retrieval via Evidential Learning

2026-07-07

AI Summary

This work addresses the challenges of cross-modal remote sensing image–text retrieval, which is highly susceptible to image degradation and textual terminological heterogeneity during testing, and where existing methods struggle to quantify query uncertainty. To this end, we introduce evidential deep learning into this task for the first time, modeling cross-modal alignment via Dirichlet distributions to estimate uncertainty. We further propose uncertainty–correctness alignment learning and intra-modal relation distillation to enhance discriminative capability. Additionally, a remote sensing–aware test-time augmentation strategy (RS-TTA) is devised, which leverages uncertainty thresholds for adaptive retrieval. The proposed method achieves state-of-the-art performance under various remote sensing–specific degradation conditions and significantly improves robustness.

Abstract

In cross-modal remote sensing image-text retrieval (CMRSITR), test-time remote sensing (RS) images and textual descriptions may deviate from well-curated benchmark conditions due to sensor- and atmosphere-related image degradations and text-side RS-vocabulary heterogeneity. Under such non-ideal conditions, existing CMRSITR methods may produce unreliable retrieval results because they perform retrieval with full certainty for each query and do not distinguish the varying uncertainty across queries. To address this issue, we propose an evidential learning-based CMRSITR (ELC) method for uncertainty-aware retrieval. During the training phase of ELC, evidential learning (EDL) is employed to model the inter-modal correspondences between RS images and textual descriptions as Dirichlet distributions, from which the uncertainty of each query can be obtained. Based on the EDL outputs, uncertainty-correctness alignment learning (UCL) is introduced to align the estimated uncertainty with retrieval correctness, encouraging high uncertainty for incorrect retrieval and low uncertainty for correct retrieval. Furthermore, intra-modal relationship learning (RL) distills the intra-modal similarity structure from pretrained mentor encoders for the trainable encoders, thereby making the Dirichlet distributions modeled by EDL more discriminative. In the test phase of ELC, the estimated uncertainty is compared with a threshold determined by a fixed deferral ratio, where low-uncertainty queries are directly returned and high-uncertainty queries are refined by RS-aware test-time augmentation (RS-TTA). Experimental results demonstrate that ELC achieves competitive retrieval performance compared with state-of-the-art CMRSITR methods and provides stronger robustness under the evaluated RS-specific degradations, including sensor- and atmosphere-related image perturbations and RS-vocabulary heterogeneity.

Problem

• cross-modal retrieval
• remote sensing
• uncertainty awareness
• image-text retrieval
• domain degradation

Innovation

• evidential learning
• uncertainty-aware retrieval
• cross-modal remote sensing
• Dirichlet distribution
• test-time augmentation