Back to Point: Exploring Point-Language Models for Zero-Shot 3D Anomaly Detection

2026-03-22

AI Summary

This work addresses the limitations of existing zero-shot 3D anomaly detection methods, which often render point clouds into 2D images and consequently lose critical geometric details, leading to insufficient sensitivity to local anomalies. To overcome this, the authors propose the BTP framework, which pioneers the use of pretrained point cloud–language models for this task. BTP aligns multi-granularity point cloud patches with textual embeddings, integrates geometric descriptors, and leverages auxiliary point cloud data for joint representation learning, thereby significantly enhancing the model’s ability to perceive and localize structural anomalies. Extensive experiments on the Real3D-AD and Anomaly-ShapeNet benchmarks demonstrate that BTP substantially outperforms current state-of-the-art methods, achieving the best-reported performance in zero-shot 3D anomaly detection.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Zero-shot (ZS) 3D anomaly detection is crucial for reliable industrial inspection, as it enables detecting and localizing defects without requiring any target-category training data. Existing approaches render 3D point clouds into 2D images and leverage pre-trained Vision-Language Models (VLMs) for anomaly detection. However, such strategies inevitably discard geometric details and exhibit limited sensitivity to local anomalies. In this paper, we revisit intrinsic 3D representations and explore the potential of pre-trained Point-Language Models (PLMs) for ZS 3D anomaly detection. We propose BTP (Back To Point), a novel framework that effectively aligns 3D point cloud and textual embeddings. Specifically, BTP aligns multi-granularity patch features with textual representations for localized anomaly detection, while incorporating geometric descriptors to enhance sensitivity to structural anomalies. Furthermore, we introduce a joint representation learning strategy that leverages auxiliary point cloud data to improve robustness and enrich anomaly semantics. Extensive experiments on Real3D-AD and Anomaly-ShapeNet demonstrate that BTP achieves superior performance in ZS 3D anomaly detection. Code will be available at \href{https://github.com/wistful-8029/BTP-3DAD}{https://github.com/wistful-8029/BTP-3DAD}.

Problem

• Zero-shot 3D anomaly detection
• Point cloud
• Anomaly localization
• Geometric details
• Industrial inspection

Innovation

• Point-Language Models
• Zero-Shot 3D Anomaly Detection
• Geometric Descriptors
• Multi-Granularity Alignment
• Joint Representation Learning