Beyond Templates: Revisiting Zero-Shot Remote Sensing through Meta-Prompting

2026-06-15

AI Summary

This work addresses the high sensitivity of vision-language models to textual prompts in zero-shot remote sensing tasks, where semantically rich descriptions generated by large language models do not consistently outperform simple templates. The authors propose a lightweight query embedding calibration strategy and employ a Meta-Prompting framework to systematically evaluate 17 models across 12 remote sensing datasets, revealing a trade-off between semantic richness and noise in the embedding space. By integrating CLIP feature whitening with text log-likelihood analysis, the proposed method significantly and consistently enhances zero-shot classification and retrieval performance without modifying model architectures, thereby demonstrating the effectiveness and practicality of embedding calibration.

Abstract

Vision-language models (VLMs) have sparked growing interest in zero-shot Earth Observation (EO) downstream tasks, with further gains enabled by remote-sensing-adapted models. We examine this setting across 17 VLM variants and 12 remote sensing (RS) datasets under Meta-Prompting for Visual Recognition (MPVR), and show that zero-shot performance remains highly sensitive to textual design choices, from the meta-prompts used to guide the LLM in generating class descriptions to the descriptions themselves. We explore why semantically rich LLM-generated class descriptions do not translate into consistent gains over simple domain-adapted CLIP-style descriptions. While LLM descriptions are more semantically expressive, they can also introduce noise in the text embedding space, reducing robustness in downstream tasks. We support this observation through a text log-likelihood analysis in the whitened CLIP feature space, comparing LLM-generated and template-based descriptions. Building on this finding, we study query embedding calibration and show that lightweight calibration of the query space consistently yields strong improvements in zero-shot classification and retrieval. Overall, our results provide practical insight into the trade-off between semantic richness and robustness, and identify embedding calibration as a simple and effective tool for improving zero-shot remote sensing performance.

Problem

• zero-shot remote sensing
• vision-language models
• textual prompt design
• embedding robustness
• semantic noise

Innovation

• zero-shot remote sensing
• vision-language models
• meta-prompting
• embedding calibration
• textual noise