A Survey on RGB, 3D, and Multimodal Approaches for Unsupervised Industrial Anomaly Detection

2024-10-29

arXiv.org

AI Summary

Existing surveys predominantly focus on unsupervised industrial anomaly detection (UIAD) using RGB modality, with insufficient coverage of 3D and RGB+3D multimodal settings. This work addresses this gap by systematically surveying task formulations, benchmark datasets, methodological approaches, and fusion strategies across all three modalities. We propose a unified taxonomy for multimodal feature fusion, encompassing point-cloud modeling, cross-modal alignment, and feature- or decision-level fusion techniques. Key common challenges—such as heterogeneous representation learning and modality imbalance—are distilled, and cross-modal representation learning is identified as a critical future direction. To enable reproducible evaluation, we establish a standardized assessment framework and release the first open-source multimodal UIAD resource repository (on GitHub), featuring a structured knowledge graph and a technology selection guide. This resource supports advancements in intelligent quality inspection and industrial digitalization.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In the advancement of industrial informatization, unsupervised anomaly detection technology effectively overcomes the scarcity of abnormal samples and significantly enhances the automation and reliability of smart manufacturing. As an important branch, industrial image anomaly detection focuses on automatically identifying visual anomalies in industrial scenarios (such as product surface defects, assembly errors, and equipment appearance anomalies) through computer vision techniques. With the rapid development of Unsupervised industrial Image Anomaly Detection (UIAD), excellent detection performance has been achieved not only in RGB setting but also in 3D and multimodal (RGB and 3D) settings. However, existing surveys primarily focus on UIAD tasks in RGB setting, with little discussion in 3D and multimodal settings. To address this gap, this artical provides a comprehensive review of UIAD tasks in the three modal settings. Specifically, we first introduce the task concept and process of UIAD. We then overview the research on UIAD in three modal settings (RGB, 3D, and multimodal), including datasets and methods, and review multimodal feature fusion strategies in multimodal setting. Finally, we summarize the main challenges faced by UIAD tasks in the three modal settings, and offer insights into future development directions, aiming to provide researchers with a comprehensive reference and offer new perspectives for the advancement of industrial informatization. Corresponding resources are available at https://github.com/Sunny5250/Awesome-Multi-Setting-UIAD.

Problem

• Reviews unsupervised industrial image anomaly detection in RGB, 3D, and multimodal settings
• Addresses lack of surveys on 3D and multimodal UIAD methods
• Summarizes challenges and future directions for multi-modal UIAD

Innovation

• Comprehensive review of RGB, 3D, multimodal UIAD
• Analyzes datasets and methods across three modalities
• Proposes future directions for industrial informatization