Conditional Visual Evidence Utility: State-Dependent Rank Reversals in Frozen Vision-Language Encoders

2026-08-28

AI Summary

研究使用冻结的视觉-语言编码器在控制组合视觉搜索中，通过条件边际效用测量方法解决了静态重要性评分无法反映证据动态价值变化的问题。

Abstract

Static importance scores compress visual evidence into a single ranking, but the value of remaining evidence can change after one cue has been observed. We study this possibility in controlled compositional visual search, where color, shape, and texture evidence can be independently exposed and their conditional marginal utility measured across acquisition states. In a held-out confirmation on 800 scenes, frozen OpenCLIP and SigLIP exhibit robust state-dependent rank reversals that concentrate in candidate-overlap regimes designed to induce ordering changes. The structure persists across two evidence-accumulation constructions and ten equivalent query wordings, but disappears under query-scene derangement. We also ask whether these reversals matter for decisions. In a post-confirmation exploratory matched-first-action analysis, reranking only after the first acquisition yields positive step-2 utility when decisions are selected under one evidence mode, wording, or backbone and evaluated under another. Together, these results show that evidence importance is state-dependent in this controlled setup and that updating an evidence ordering can retain decision-relevant value across evaluator changes. They motivate evaluating vision-language evidence use conditionally rather than through a single static ranking, while providing a measurable target for future adaptive evidence-selection methods.

Problem

• Conditional Visual Evidence
• State-Dependent Rank Reversals
• Frozen Vision-Language Encoders
• Compositional Visual Search
• Evidence Importance

Innovation

• Conditional Visual Evidence Utility
• State-Dependent Rank Reversals
• Frozen Vision-Language Encoders
• Evidence Accumulation
• Decision-Relevant Value