Finding Dino: A plug-and-play framework for unsupervised detection of out-of-distribution objects using prototypes

2024-04-11

arXiv.org

AI Summary

To address the challenge of detecting and localizing unknown or out-of-distribution (OOD) objects in safety-critical autonomous driving scenarios—without any annotations or prior knowledge of OOD categories—we propose PROWL: the first zero-shot, unsupervised, cross-domain generalizable framework for anomalous object detection and localization. PROWL leverages prototype features extracted from self-supervised vision models (e.g., DINO), integrates class-agnostic saliency modeling, and introduces a zero-shot category-guided mechanism—requiring neither target-domain labels, fine-tuning, nor auxiliary OOD data. The method is plug-and-play and generalizes across diverse domains, including road, rail, and maritime scenes. On RoadAnomaly and RoadObstacle benchmarks, PROWL achieves state-of-the-art performance, matching supervised methods that rely on OOD annotations. This significantly advances the practicality, robustness, and domain-agnostic capability of OOD detection in real-world autonomous systems.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Unsupervised & Self-Supervised Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Data transparency and provenance

Abstract

Detecting and localising unknown or out-of-distribution (OOD) objects in any scene can be a challenging task in vision, particularly in safety-critical cases involving autonomous systems like automated vehicles or trains. Supervised anomaly segmentation or open-world object detection models depend on training on exhaustively annotated datasets for every domain and still struggle in distinguishing between background and OOD objects. In this work, we present a plug-and-play framework - PRototype-based OOD detection Without Labels (PROWL). It is an inference-based method that does not require training on the domain dataset and relies on extracting relevant features from self-supervised pre-trained models. PROWL can be easily adapted to detect in-domain objects in any operational design domain (ODD) in a zero-shot manner by specifying a list of known classes from this domain. PROWL, as a first zero-shot unsupervised method, achieves state-of-the-art results on the RoadAnomaly and RoadObstacle datasets provided in road driving benchmarks - SegmentMeIfYouCan (SMIYC) and Fishyscapes, as well as comparable performance against existing supervised methods trained without auxiliary OOD data. We also demonstrate its generalisability to other domains such as rail and maritime.

Problem

• Detecting unknown out-of-distribution objects
• Zero-shot unsupervised object detection
• Generalizable across multiple operational domains

Innovation

• Zero-shot OOD detection
• Self-supervised pre-trained models
• Plug-and-play framework PROWL