GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models

2026-07-01

AI Summary

Industrial quality inspection demands a unified approach capable of simultaneously performing anomaly detection, defect localization, type identification, and explainable analysis—capabilities that existing methods struggle to integrate effectively. This work proposes GenAU, the first vision-language unified framework that jointly leverages language understanding and pixel-level anomaly segmentation. By introducing dedicated segmentation tokens [SEG_defect] and [SEG_normal], GenAU enables language-guided defect localization and supports zero-shot multi-type defect detection along with structured semantic descriptions. The method integrates multi-scale visual features, instruction tuning, and a dual-segmentation-token mechanism, achieving superior image-level detection performance over existing CLIP-based zero-shot approaches on cross-dataset benchmarks VisA and Real-IAD, while attaining segmentation accuracy comparable to specialized CLIP baselines.

Abstract

Industrial inspection requires more than binary anomaly detection: a practical system should determine whether an anomaly exists, localize the defective region, identify the defect type, and provide interpretable visual evidence. Existing CLIP-based methods detect and localize anomalies well but offer limited language-level defect understanding, while instruction-tuned vision-language models can describe defects but do not natively produce pixel-level masks. We introduce GenAU, a Generalist vision-language framework for industrial Anomaly Understanding that unifies image-level detection, pixel-level segmentation, multi-type anomaly detection, and defect analysis in a single instruction-following model. GenAU augments a vision-language model with two segmentation tokens, [SEG_defect] and [SEG_normal], whose hidden states act as language-grounded queries over multi-scale visual features for pixel-level localization; the image-level score fuses this map with the decoder's textual normal/defect decision, while the language decoder produces structured defect-aware responses. Trained with a joint language-modeling and segmentation objective, GenAU covers all four tasks within one architecture and recipe, adding zero-shot multi-type detection and language-grounded defect analysis at a quantified cost to detection and segmentation. Across cross-dataset benchmarks, GenAU attains the strongest image-level detection among CLIP-based zero-shot methods on VisA and Real-IAD, with segmentation approaching but not surpassing specialized CLIP baselines.

Problem

• industrial anomaly understanding
• vision-language models
• anomaly detection
• defect localization
• interpretable defect analysis

Innovation

• vision-language model
• anomaly understanding
• pixel-level segmentation
• instruction-following
• language-grounded detection