ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images

2026-06-27

AI Summary

This work addresses the modality gap between natural language descriptions and fine-grained visual cues in remote sensing imagery, which often leads to localization drift. To tackle this issue, the authors propose a training-free visual grounding framework that leverages a one-shot visual prompting mechanism. Specifically, a Visual Example Calibrator (VEC) refines the coarse cross-modal priors generated by multimodal large language models, while a Structure-Aware Refiner (SAR) produces high-quality geometric prompts to guide SAM for precise pixel-level segmentation. This example-driven calibration-and-refinement pipeline effectively bridges the semantic gap between language and remote sensing images, significantly outperforming existing training-free and weakly supervised methods across multiple benchmarks. The approach markedly improves boundary localization accuracy in complex scenes and suppresses background interference.

Abstract

Remote sensing visual grounding (RSVG) aims to locate specific objects in high-resolution RS imagery using free-form natural language descriptions. While recent advances in multimodal large language models (MLLMs) show great potential for such open-vocabulary RSVG, their training-free adaptation is hindered by the modality gap between abstract linguistic semantics and fine-grained visual cues. In cluttered RS scenes, this gap inevitably causes severe localization drift. To bridge this gap, we propose Exemplar-driven Calibrated Refinement (ExACT), a novel training-free framework driven by a one-shot visual prompting mechanism to explicitly provide discriminative structural guidance for precise pixel-level localization. Specifically, we propose a Vision Exemplar-based Calibrator (VEC) that extracts fine-grained visual correspondences from the given exemplar to rectify the rough cross-modal priors from frozen MLLMs, effectively suppressing background artifacts and accurately outlining target boundaries. Subsequently, a Structure-Aware Refiner (SAR) employs an iterative merge-and-select clustering strategy to consolidate the calibrated priors into high-quality positive and negative geometric prompts. These prompts then guide the Segment Anything Model (SAM) to achieve precise pixel-level predictions. Extensive experiments confirm the superiority of ExACT over existing training-free and weakly-supervised methods.

Problem

• visual grounding
• remote sensing
• modality gap
• localization drift
• multimodal large language models

Innovation

• training-free visual grounding
• exemplar-driven refinement
• multimodal large language models
• pixel-level localization
• remote sensing imagery