🤖 AI Summary
This study investigates whether the effectiveness of frozen vision-language model prompts—such as those from CLIP—remains consistent after visual adaptation to a target domain in unsupervised cross-domain few-shot learning. By evaluating performance shifts of simple class-name templates versus rich semantic descriptions before and after adaptation across multiple medical and remote sensing datasets, and employing LoRA fine-tuning, paired comparisons, semantic shuffling controls, and multi-seed validation, the work uncovers two novel phenomena: “semantic saturation,” where adaptation yields diminishing gains (e.g., EuroSAT, CropDisease), and “semantic emergence,” where only detailed descriptions become effective post-adaptation (e.g., ISIC, ChestX). These findings challenge the prevailing assumption that zero-shot prompt quality reliably predicts its efficacy as a semantic anchor after adaptation, revealing instead a dynamic evolution of semantic utility.
📝 Abstract
Language descriptions in source-free cross-domain few-shot learning (SF-CDFSL) are often selected according to zero-shot accuracy obtained with a frozen vision--language model. This paper asks whether that ranking remains valid after target-domain visual adaptation. Under a strictly paired protocol, we compare a generic class-name template with fixed detailed class descriptions before and after visual Low-Rank Adaptation (LoRA) on EuroSAT, CropDisease, ISIC, and ChestX. Let $\deltazero$ and $\deltalora$ denote the Detailed-minus-Base accuracy before and after adaptation, respectively. Two recurring regimes emerge. In \emph{semantic saturation}, $\deltazero>0$ but $0<\deltalora\ll\deltazero$: on EuroSAT and CropDisease, initial gains of 8.13--21.54 percentage points contract to 0.69--2.96 points after LoRA. In \emph{semantic emergence}, $\deltazero\leq0$ but $\deltalora>0$: on ISIC and ChestX, detailed descriptions become more useful only after the visual representation is updated. Training trajectories and sample-level decomposition show that saturation is driven mainly by Base-LoRA recovering errors already solved by detailed semantics, whereas emergence is associated with prediction turnover and newly formed Detailed-only correct decisions. Fixed-point-free shuffled-semantic controls, a second CLIP backbone, and multiple random seeds support the broad pattern while identifying ChestX 1-shot as a weak boundary case. These findings establish that zero-shot prompt quality is an incomplete proxy for adaptation-anchor quality and motivate evaluating language on both sides of the adaptation boundary.