Problem
Research questions and friction points this paper is trying to address.
one-shot exemplar-conditioned object detection
cluttered scenes
world-model pretraining
object localization
target presence
WALDO: One-Shot Exemplar-Conditioned Object Detection in Cluttered Scenes
🤖 AI Summary
本文提出WALDO模型，利用少量参数和预训练特征，在杂乱场景中通过单个参考图像和简短描述定位特定对象，并判断目标是否存在。
📝 Abstract
Locating a specific object instance in a cluttered scene using a single reference image and a short description, and reporting when that instance is absent, large vision-language models usually address this task. We ask whether the same capability is available far more cheaply, from representations already learned by a world-model pretraining objective. We present WALDO, a one-shot exemplar- and language-conditioned detection head with 3.4M trainable parameters that reads frozen V-JEPA 2.1 features to jointly predict object localization and target presence, with no gradient on the backbone. Because exemplar-conditioned supervision is scarce, we synthesize training episodes from instance annotations, mining exemplars from ground-truth boxes and constructing absence cases that exclude the referenced instance while leaving same-category distractors in view. This is easy to get wrong: in the obvious implementation, crop size alone predicts the label, and a head trained on it reaches 0.9998 absence AUROC without ever consulting the exemplar, and we report the negative controls that close the shortcut. On 35 held-out cluttered scenes, WALDO achieves a 0.461 catalogue AP@50, compared to 0.306 for a prompted Grounding DINO baseline under an identical scorer. Substituting DINOv3 for V-JEPA under a matched 576-token grid drops within-category absence AUROC from 0.880 to 0.726 and instance AP@50 from 0.201 to 0.141, isolating the pretraining objective rather than input resolution as the source of the gain. Instance-level Success@1, however, reaches only 0.190 against a 0.190 category-chance floor: world-model features transfer to localization precision and absence detection but not to instance identity.
Innovation
Methods, ideas, or system contributions that make the work stand out.
one-shot exemplar-conditioned detection
world-model pretraining
synthesizing training episodes
V-JEPA features
absence detection
🔎 Similar Papers
No similar papers found.
