Look Inside for More: Internal Spatial Modality Perception for 3D Anomaly Detection

2024-12-18

arXiv.org

AI Summary

Existing 3D anomaly detection methods predominantly rely on external geometric structures of point clouds while neglecting internal spatial information, leading to limited representational capacity. To address this, we propose the first anomaly detection paradigm explicitly targeting the *internal structure* of point clouds, introducing a theoretically grounded Internal Spatial Modality Perception (ISMP) framework. At its core lies the Spatial Insight Engine (SIE), a global abstraction module that jointly performs keypoint feature enhancement guided by internal sampling and adaptive feature filtering, enabling explicit alignment between internal and external structural representations. Evaluated on the Real3D-AD benchmark, our method achieves +4.2% AUROC improvement at the object level and +13.1% at the pixel level. Moreover, the SIE module demonstrates strong cross-task generalization—e.g., in classification and segmentation—validating both the effectiveness and broad applicability of internal spatial modality modeling for 3D perception.

Technology Category

• Computer Vision: 3D Computer Vision
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

3D anomaly detection has recently become a significant focus in computer vision. Several advanced methods have achieved satisfying anomaly detection performance. However, they typically concentrate on the external structure of 3D samples and struggle to leverage the internal information embedded within samples. Inspired by the basic intuition of why not look inside for more, we introduce a straightforward method named Internal Spatial Modality Perception (ISMP) to explore the feature representation from internal views fully. Specifically, our proposed ISMP consists of a critical perception module, Spatial Insight Engine (SIE), which abstracts complex internal information of point clouds into essential global features. Besides, to better align structural information with point data, we propose an enhanced key point feature extraction module for amplifying spatial structure feature representation. Simultaneously, a novel feature filtering module is incorporated to reduce noise and redundant features for further aligning precise spatial structure. Extensive experiments validate the effectiveness of our proposed method, achieving object-level and pixel-level AUROC improvements of 4.2% and 13.1%, respectively, on the Real3D-AD benchmarks. Note that the strong generalization ability of SIE has been theoretically proven and is verified in both classification and segmentation tasks.

Problem

• Enhance 3D anomaly detection by leveraging internal spatial information.
• Develop ISMP method to extract global features from internal views.
• Improve AUROC in object-level and pixel-level anomaly detection benchmarks.

Innovation

• Internal Spatial Modality Perception (ISMP) for 3D anomaly detection
• Spatial Insight Engine (SIE) abstracts internal point cloud information
• Enhanced key point feature extraction and noise reduction modules