iEBAKER: Improved Remote Sensing Image-Text Retrieval Framework via Eliminate Before Align and Keyword Explicit Reasoning

2025-04-08

AI Summary

To address cross-modal embedding bias in remote sensing image–text retrieval—caused by interference from weakly relevant samples and insufficient discrimination of textual key concepts—this paper proposes an end-to-end foundation model adaptation framework. Methodologically, it introduces three core innovations: (1) an explicit “Eliminate-then-Align” (EBA) strategy to filter out weakly relevant image–text pairs; (2) a Keyword Explicit Reasoning (KER) module to enhance fine-grained textual semantic modeling; and (3) a Symmetric Alignment-based Re-ranking (SAR) mechanism that decouples local and global similarity estimation. Crucially, the framework requires no remote sensing–specific pretraining and directly adapts general-purpose foundation models (e.g., CLIP). Evaluated on three major remote sensing benchmarks, it consistently surpasses state-of-the-art methods, achieving significant gains in retrieval accuracy while reducing training data requirements by 30%.

Technology Category

• Computer Vision: Image and Video Retrieval
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Recent studies focus on the Remote Sensing Image-Text Retrieval (RSITR), which aims at searching for the corresponding targets based on the given query. Among these efforts, the application of Foundation Models (FMs), such as CLIP, to the domain of remote sensing has yielded encouraging outcomes. However, existing FM based methodologies neglect the negative impact of weakly correlated sample pairs and fail to account for the key distinctions among remote sensing texts, leading to biased and superficial exploration of sample pairs. To address these challenges, we propose an approach named iEBAKER (an Improved Eliminate Before Align strategy with Keyword Explicit Reasoning framework) for RSITR. Specifically, we propose an innovative Eliminate Before Align (EBA) strategy to filter out the weakly correlated sample pairs, thereby mitigating their deviations from optimal embedding space during alignment.Further, two specific schemes are introduced from the perspective of whether local similarity and global similarity affect each other. On this basis, we introduce an alternative Sort After Reversed Retrieval (SAR) strategy, aims at optimizing the similarity matrix via reverse retrieval. Additionally, we incorporate a Keyword Explicit Reasoning (KER) module to facilitate the beneficial impact of subtle key concept distinctions. Without bells and whistles, our approach enables a direct transition from FM to RSITR task, eliminating the need for additional pretraining on remote sensing data. Extensive experiments conducted on three popular benchmark datasets demonstrate that our proposed iEBAKER method surpasses the state-of-the-art models while requiring less training data. Our source code will be released at https://github.com/zhangy0822/iEBAKER.

Problem

• Eliminate weakly correlated sample pairs in RSITR
• Address key distinctions among remote sensing texts
• Optimize similarity matrix via reverse retrieval

Innovation

• Eliminate Before Align strategy filters weakly correlated pairs
• Sort After Reversed Retrieval optimizes similarity matrix
• Keyword Explicit Reasoning enhances key concept distinctions