Real-IAD Variety: Pushing Industrial Anomaly Detection Dataset to a Modern Era

2025-11-01

AI Summary

Existing industrial anomaly detection (IAD) research is hindered by limited public benchmarks—characterized by narrow category coverage and insufficient scale—leading to evaluation saturation and poor model generalization. To address this, we introduce Real-IAD Variety, the largest and most diverse IAD benchmark to date: it encompasses 160 object categories, 28 industrial domains, 24 material types, and 22 color variations, with 198,960 high-resolution images. We propose the first comprehensive evaluation protocol supporting multi-category unsupervised learning, multi-view analysis, and zero-/few-shot settings. Extensive experiments reveal that state-of-the-art methods suffer substantial performance degradation under category expansion, whereas vision-language models demonstrate superior robustness and scalability. Real-IAD Variety establishes a rigorous, realistic benchmark for developing and evaluating foundation models for general-purpose IAD.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Adversarial Attacks & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Industrial Anomaly Detection (IAD) is critical for enhancing operational safety, ensuring product quality, and optimizing manufacturing efficiency across global industries. However, the IAD algorithms are severely constrained by the limitations of existing public benchmarks. Current datasets exhibit restricted category diversity and insufficient scale, frequently resulting in metric saturation and limited model transferability to real-world scenarios. To address this gap, we introduce Real-IAD Variety, the largest and most diverse IAD benchmark, comprising 198,960 high-resolution images across 160 distinct object categories. Its diversity is ensured through comprehensive coverage of 28 industries, 24 material types, and 22 color variations. Our comprehensive experimental analysis validates the benchmark's substantial challenge: state-of-the-art multi-class unsupervised anomaly detection methods experience significant performance degradation when scaled from 30 to 160 categories. Crucially, we demonstrate that vision-language models exhibit remarkable robustness to category scale-up, with minimal performance variation across different category counts, significantly enhancing generalization capabilities in diverse industrial contexts. The unprecedented scale and complexity of Real-IAD Variety position it as an essential resource for training and evaluating next-generation foundation models for anomaly detection. By providing this comprehensive benchmark with rigorous evaluation protocols across multi-class unsupervised, multi-view, and zero-/few-shot settings, we aim to accelerate research beyond domain-specific constraints, enabling the development of scalable, general-purpose anomaly detection systems. Real-IAD Variety will be made publicly available to facilitate innovation in this critical field.

Problem

• Existing industrial anomaly detection datasets lack category diversity and scale
• Current benchmarks cause metric saturation and limited real-world transferability
• Vision-language models need evaluation on comprehensive multi-category industrial datasets

Innovation

• Largest industrial anomaly detection benchmark with 198,960 images
• Covers 160 object categories across 28 industries
• Vision-language models enhance generalization across categories