UOD: Unseen Object Detection in 3D Point Cloud

2024-01-08

AI Summary

Detecting out-of-distribution (OOD) objects in open-world autonomous driving scenarios remains challenging due to the lack of 3D point cloud detectors capable of localizing and identifying unknown-category objects. Method: This paper introduces the first general-purpose 3D OOD detection framework, featuring a novel universal objectness modeling scheme that jointly learns object localization and OOD classification; it further incorporates anomaly sample augmentation and end-to-end joint optimization to eliminate reliance on predefined categories. Contribution/Results: We establish the first real-world KITTI Misc benchmark and two synthetic OOD benchmarks (nuScenes OOD and SUN-RGBD OOD). Extensive experiments across multiple baseline detectors demonstrate significant improvements in OOD recall and classification accuracy, establishing a new paradigm for open-set 3D OOD detection in unstructured environments.

Technology Category

• Computer Vision: Object Detection & Categorization
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Existing 3D object detectors encounter extreme challenges in localizing unseen 3D objects and recognizing them as unseen, which is a crucial technology in autonomous driving in the wild. To address these challenges, we propose practical methods to enhance the performance of 3D detection and Out-Of-Distribution (OOD) classification for unseen objects. The proposed methods include anomaly sample augmentation, learning of universal objectness, learning of detecting unseen objects, and learning of distinguishing unseen objects. To demonstrate the effectiveness of our approach, we propose the KITTI Misc benchmark and two additional synthetic OOD benchmarks: the Nuscenes OOD benchmark and the SUN-RGBD OOD benchmark. The proposed methods consistently enhance performance by a large margin across all existing methods, giving insight for future work on unseen 3D object detection in the wild.

Problem

• Detecting unseen 3D objects in point clouds
• Improving 3D detection and OOD classification
• Enhancing performance via novel learning methods

Innovation

• Anomaly sample augmentation for unseen objects
• Universal objectness learning in 3D detection
• Distinguishing unseen objects via OOD classification