Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection

2026-04-25

AI Summary

This work addresses two key challenges in open-vocabulary object detection: inaccurate region-level pseudo-label assignment from vision-language models and biased, unreliable objectness scores from region proposal networks for novel categories. To tackle these issues, the authors propose a novel pseudo-labeling framework featuring a hierarchical confidence calibration mechanism that enforces class consistency across semantic levels. They also introduce a parameter-efficient LoCLIP model augmented with an objectness token to mitigate the bias of region proposals toward base classes. By integrating an enhanced CLIP architecture, hierarchical semantic consistency analysis, and region-aware pseudo-label generation, the method achieves state-of-the-art performance on standard benchmarks such as COCO and LVIS, significantly outperforming existing approaches.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Conventional object detectors typically operate under a closed-set assumption, limiting recognition to a predefined set of base classes seen during training. Open-vocabulary object detection (OVD) addresses this limitation by leveraging vision-language models (VLMs) to generate pseudo labels for novel object classes. However, existing OVD methods suffer from two critical drawbacks: (1) inaccurate class label assignments, as VLMs are optimized for image-level predictions rather than the region-level predictions required for pseudo labeling, and (2) unreliable objectness scores from region proposal networks (RPNs) trained exclusively on base object classes. To address these issues, we propose a novel pseudo labeling framework for OVD. Our approach introduces a hierarchical confidence calibration (HCC) technique, which ensures reliable class label estimation by assessing consistency across hierarchical semantic levels (class, super- and sub-category). We also present LoCLIP, a parameter-efficient adaptation of CLIP that incorporates an objectness token to mitigate base class bias problem of RPNs and provide reliable objectness estimations for novel object classes. Extensive experiments on standard OVD benchmarks, including COCO and LVIS, demonstrate that our approach clearly sets a new state of the art, validating the effectiveness of our approach. Project site: https://cvlab.yonsei.ac.kr/projects/HCC

Problem

• open-vocabulary object detection
• pseudo labeling
• vision-language models
• objectness estimation
• class label assignment

Innovation

• Hierarchical Consistency
• Unbiased Objectness
• Open-Vocabulary Object Detection
• Pseudo Labeling
• LoCLIP