OPUS: A Simple yet Effective Unified Framework for Open-Vocabulary Detection

📅 2026-08-31
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
本文提出OPUS框架,通过语义丰富的视觉表示和可扩展的定位监督简化开放词汇检测,支持多种提示方式并在多项基准测试中取得领先性能。
📝 Abstract
Recent unified open-vocabulary detection (OVD) supports heterogeneous prompts, including text queries, visual exemplars, and their combinations, but often rely on increasingly complex designs such as heavy cross-modal fusion, staged training, and iterative annotation pipelines. We revisit whether such complexity is necessary in the era of stronger foundation models. Our finding is that unified OVD can be made substantially simpler with semantic-rich visual representations and scalable grounding supervision. We present OPUS (\textbf{O}pen-vocabulary, \textbf{P}rompt-\textbf{U}nified, \textbf{S}imple), a unified detector supporting text, interactive visual, generic visual, and mixed prompting within one framework. OPUS adopts a simple three-part design. Its model architecture combines a semantic-rich visual encoder, built on a DINOv3-ConvNeXt-B backbone with efficient hybrid encoding, with a prompt-aware decoder that avoids prompt-specific branches for unified prompt reasoning. OPUS is trained with a one-stage text-visual training strategy with Instance-level Contrastive Alignment (ICA), and is supported by a SAM3-based single-pass data engine for heterogeneous grounding supervision. Experiments on COCO, LVIS-minival, and ODinW35 show that OPUS achieves state-of-the-art Visual-I performance, reaching 68.1/69.2/54.7 AP, while maintaining balanced Text and Visual-G accuracy. OPUS also turns mixed prompting from interference into complementarity, improving over text or visual prompt alone. These results show that simplicity and strong unified prompting capability can be achieved together.
Problem

Research questions and friction points this paper is trying to address.

unified open-vocabulary detection
heterogeneous prompts
cross-modal fusion
Innovation

Methods, ideas, or system contributions that make the work stand out.

Unified Open-Vocabulary Detection
Semantic-Rich Visual Representations
Scalable Grounding Supervision
Prompt-Aware Decoder
Instance-level Contrastive Alignment
X
Xiaoyan Wei
Megvii Technology Inc.
Z
Zhimin Yao
Megvii Technology Inc.
R
Ruilin Yang
Megvii Technology Inc.
W
Wei Zhang
Megvii Technology Inc.
Y
Yong Dai
X-Humanoid
Y
Yi Zhang
X-Humanoid
W
Wei Ge
Megvii Technology Inc.