Exploiting Vision Language Model for Training-Free 3D Point Cloud OOD Detection via Graph Score Propagation

2025-06-27

AI Summary

Out-of-distribution (OOD) detection in 3D point clouds remains a critical challenge for safety-critical perception systems, particularly due to the absence of labeled OOD data and the difficulty of generalizing across diverse real-world scenes. Method: We propose the first training-free, vision-language model (VLM)-driven framework for 3D OOD detection. Our approach constructs class prototypes and integrates prompt clustering with self-trained negative prompt generation. Crucially, it introduces graph-structured modeling and score propagation to enforce cross-sample semantic consistency in an unsupervised manner, enabling precise anomaly localization without any OOD supervision or fine-tuning. Contribution/Results: The framework supports few-shot adaptation and eliminates reliance on OOD samples or parameter updates. Evaluated on both synthetic and real-world benchmarks—including SemanticKITTI and Waymo—it surpasses state-of-the-art methods, achieving an average AUC improvement of 5.2% for OOD detection. This work establishes the first empirical validation of VLMs’ effectiveness and generalizability in training-free 3D OOD detection.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Out-of-distribution (OOD) detection in 3D point cloud data remains a challenge, particularly in applications where safe and robust perception is critical. While existing OOD detection methods have shown progress for 2D image data, extending these to 3D environments involves unique obstacles. This paper introduces a training-free framework that leverages Vision-Language Models (VLMs) for effective OOD detection in 3D point clouds. By constructing a graph based on class prototypes and testing data, we exploit the data manifold structure to enhancing the effectiveness of VLMs for 3D OOD detection. We propose a novel Graph Score Propagation (GSP) method that incorporates prompt clustering and self-training negative prompting to improve OOD scoring with VLM. Our method is also adaptable to few-shot scenarios, providing options for practical applications. We demonstrate that GSP consistently outperforms state-of-the-art methods across synthetic and real-world datasets 3D point cloud OOD detection.

Problem

• Detecting OOD in 3D point clouds without training
• Leveraging VLMs for 3D OOD detection challenges
• Improving OOD scoring via Graph Score Propagation

Innovation

• Training-free 3D OOD detection via VLMs
• Graph Score Propagation with prompt clustering
• Adaptable to few-shot scenarios effectively