From Lab to Factory: Pitfalls and Guidelines for Self-/Unsupervised Defect Detection on Low-Quality Industrial Images

2025-06-20

AI Summary

Industrial metal sandblasted forgings often exhibit minute surface defects, yet existing unsupervised defect detection methods suffer from poor robustness and distorted evaluation under low-quality RGB images prevalent in real production lines. Method: This paper proposes the first unsupervised defect detection and diagnostic framework tailored to authentic industrial scenarios. It systematically analyzes the data–model coupling failure mechanism, exposing intrinsic limitations of likelihood-based approaches on degraded imagery; introduces perturbation-robustness testing, invariance diagnostics, and empirical AUROC limitation analysis to establish a more reliable empirical risk assessment paradigm; and conducts comparative validation using representative reconstruction-based (e.g., VAE) and embedding-based (e.g., PatchCore) models. Contribution/Results: The framework delivers reusable diagnostic guidelines and actionable improvement pathways, significantly enhancing defect localization reliability and deployment stability on real-world production-line data.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

The detection and localization of quality-related problems in industrially mass-produced products has historically relied on manual inspection, which is costly and error-prone. Machine learning has the potential to replace manual handling. As such, the desire is to facilitate an unsupervised (or self-supervised) approach, as it is often impossible to specify all conceivable defects ahead of time. A plethora of prior works have demonstrated the aptitude of common reconstruction-, embedding-, and synthesis-based methods in laboratory settings. However, in practice, we observe that most methods do not handle low data quality well or exude low robustness in unfavorable, but typical real-world settings. For practitioners it may be very difficult to identify the actual underlying problem when such methods underperform. Worse, often-reported metrics (e.g., AUROC) are rarely suitable in practice and may give misleading results. In our setting, we attempt to identify subtle anomalies on the surface of blasted forged metal parts, using rather low-quality RGB imagery only, which is a common industrial setting. We specifically evaluate two types of state-of-the-art models that allow us to identify and improve quality issues in production data, without having to obtain new data. Our contribution is to provide guardrails for practitioners that allow them to identify problems related to, e.g., (lack of) robustness or invariance, in either the chosen model or the data reliably in similar scenarios. Furthermore, we exemplify common pitfalls in and shortcomings of likelihood-based approaches and outline a framework for proper empirical risk estimation that is more suitable for real-world scenarios.

Problem

• Unsupervised defect detection on low-quality industrial images
• Handling poor robustness in real-world industrial settings
• Improving defect identification without requiring new data

Innovation

• Unsupervised defect detection on low-quality images
• Robust model evaluation in real-world settings
• Framework for empirical risk estimation improvement