Open World Object Detection: A Survey

2024-10-15

IEEE transactions on circuits and systems for video technology (Print)

AI Summary

Open-World Object Detection (OWOD) aims to enable models to simultaneously detect unknown classes, mitigate catastrophic forgetting, and dynamically expand their knowledge boundaries while continuously learning novel categories. This paper presents the first comprehensive, panoramic survey of OWOD, establishing a unified problem formulation, evaluation protocol, and benchmark dataset suite, while rigorously delineating its theoretical distinctions from open-set recognition and incremental learning. We systematically analyze over 100 state-of-the-art works, covering paradigms including class-incremental OWOD, pseudo-labeling, and memory replay, as well as architectural adaptations for detectors such as Faster R-CNN and DETR. Furthermore, we release the first unified codebase and benchmarking platform. Quantitative evaluation reveals persistent bottlenecks across methods in unknown-class detection recall, known-class retention stability, and cross-stage generalization performance—providing a standardized foundation and principled guidance for future research.

Technology Category

• Computer Vision: Object Detection & Categorization
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Exploring new knowledge is a fundamental human ability that can be mirrored in the development of deep neural networks, especially in the field of object detection. Open world object detection (OWOD) is an emerging area of research that adapts this principle to explore new knowledge. It focuses on recognizing and learning from objects absent from initial training sets, thereby incrementally expanding its knowledge base when new class labels are introduced. This survey paper offers a thorough review of the OWOD domain, covering essential aspects, including problem definitions, benchmark datasets, source codes, evaluation metrics, and a comparative study of existing methods. Additionally, we investigate related areas like open set recognition (OSR) and incremental learning (IL), underlining their relevance to OWOD. Finally, the paper concludes by addressing the limitations and challenges faced by current OWOD algorithms and proposes directions for future research. To our knowledge, this is the first comprehensive survey of the emerging OWOD field with over one hundred references, marking a significant step forward for object detection technology. A comprehensive source code and benchmarks are archived and concluded at https://github.com/ArminLee/OWOD_Review.

Problem

• Recognizing and learning from unseen object classes
• Incrementally expanding knowledge with new class labels
• Reviewing benchmarks and methods for open world detection

Innovation

• Recognizes and learns from unknown objects
• Incrementally expands knowledge with new classes
• Integrates open set recognition and incremental learning