MAU-GPT: Enhancing Multi-type Industrial Anomaly Understanding via Anomaly-aware and Generalist Experts Adaptation

2026-01-31

AI Summary

This work addresses the limitations of existing industrial defect detection methods, which suffer from insufficient data coverage and poor generalization under complex anomaly patterns, hindering fine-grained and scalable automated quality inspection. To overcome these challenges, we propose MAU-GPT, a multimodal large language model incorporating a novel Anomaly-aware Mixture of Experts with LoRA (AMoE-LoRA) mechanism. This approach enables parameter-efficient fine-tuning through adaptive collaboration between anomaly-aware and general-purpose experts, supporting multi-level tasks ranging from binary classification to complex reasoning. We also introduce MAU-Set, the first hierarchical benchmark dataset and evaluation protocol tailored for cross-domain industrial anomaly understanding. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches across diverse industrial scenarios, achieving superior performance in detecting and reasoning about unseen and intricate defects, thereby validating its scalability and practical utility.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Multi-modal Vision

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

As industrial manufacturing scales, automating fine-grained product image analysis has become critical for quality control. However, existing approaches are hindered by limited dataset coverage and poor model generalization across diverse and complex anomaly patterns. To address these challenges, we introduce MAU-Set, a comprehensive dataset for Multi-type industrial Anomaly Understanding. It spans multiple industrial domains and features a hierarchical task structure, ranging from binary classification to complex reasoning. Alongside this dataset, we establish a rigorous evaluation protocol to facilitate fair and comprehensive model assessment. Building upon this foundation, we further present MAU-GPT, a domain-adapted multimodal large model specifically designed for industrial anomaly understanding. It incorporates a novel AMoE-LoRA mechanism that unifies anomaly-aware and generalist experts adaptation, enhancing both understanding and reasoning across diverse defect classes. Extensive experiments show that MAU-GPT consistently outperforms prior state-of-the-art methods across all domains, demonstrating strong potential for scalable and automated industrial inspection.

Problem

• industrial anomaly understanding
• dataset coverage
• model generalization
• anomaly patterns
• quality control

Innovation

• MAU-GPT
• AMoE-LoRA
• industrial anomaly understanding
• multimodal large model
• expert adaptation