OmniAD: Detect and Understand Industrial Anomaly via Multimodal Reasoning

2025-05-28

AI Summary

Industrial anomaly analysis faces challenges including coarse-grained localization, limited interpretability, and insufficient integration of domain knowledge. To address these, we propose the first vision–text dual-modal collaborative analysis framework tailored for industrial scenarios. Our method introduces (1) Text-as-Mask encoding—a novel, threshold-free, generative anomaly detection mechanism; (2) a vision-guided textual reasoning module enabling precise anomaly localization and semantically grounded attribution; and (3) a triple-reward training paradigm integrating supervised fine-tuning (SFT) and GRPO-based reinforcement learning to enhance knowledge-augmented analysis. Evaluated on the MMAD benchmark, our framework achieves 79.1 points, substantially outperforming Qwen2.5-VL-7B and GPT-4o. Moreover, it establishes new state-of-the-art results across multiple industrial anomaly detection benchmarks, demonstrating superior generalization and interpretability in real-world settings.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

While anomaly detection has made significant progress, generating detailed analyses that incorporate industrial knowledge remains a challenge. To address this gap, we introduce OmniAD, a novel framework that unifies anomaly detection and understanding for fine-grained analysis. OmniAD is a multimodal reasoner that combines visual and textual reasoning processes. The visual reasoning provides detailed inspection by leveraging Text-as-Mask Encoding to perform anomaly detection through text generation without manually selected thresholds. Following this, Visual Guided Textual Reasoning conducts comprehensive analysis by integrating visual perception. To enhance few-shot generalization, we employ an integrated training strategy that combines supervised fine-tuning (SFT) with reinforcement learning (GRPO), incorporating three sophisticated reward functions. Experimental results demonstrate that OmniAD achieves a performance of 79.1 on the MMAD benchmark, surpassing models such as Qwen2.5-VL-7B and GPT-4o. It also shows strong results across multiple anomaly detection benchmarks. These results highlight the importance of enhancing visual perception for effective reasoning in anomaly understanding. All codes and models will be publicly available.

Problem

• Detect and analyze industrial anomalies via multimodal reasoning
• Integrate visual and textual reasoning for fine-grained anomaly understanding
• Improve few-shot generalization with combined SFT and GRPO training

Innovation

• Multimodal reasoning for anomaly detection
• Text-as-Mask Encoding for threshold-free detection
• Integrated SFT and GRPO training strategy