Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation

2025-11-28

AI Summary

In semi-controlled industrial environments lacking historical defect data, deploying robust visual inspection systems remains challenging. To address this, we propose a zero-shot, pose-invariant multi-criteria visual quality inspection framework. Our method unifies pose modeling and structured defect definition by integrating semantic scene descriptions with hierarchical annotation strategies. Leveraging CAD models, we enable real-time construction and dynamic mapping of RGB-D digital twins; defects are localized and evaluated via 3D distance metrics and differentiable rendering—eliminating the need for ground-truth masks. Key innovations include zero-shot defect detection without annotated defect samples, explicit decoupling of pose estimation from defect localization, and support for multi-criteria, general-purpose inspection. Evaluated on axial-flux motor quality inspection, our approach achieves 63.3% IoU using only lightweight distance-based metrics, enabling real-time inference while substantially reducing data acquisition and deployment overhead.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Early-stage visual quality inspection is vital for achieving Zero-Defect Manufacturing and minimizing production waste in modern industrial environments. However, the complexity of robust visual inspection systems and their extensive data requirements hinder widespread adoption in semi-controlled industrial settings. In this context, we propose a pose-agnostic, zero-shot quality inspection framework that compares real scenes against real-time Digital Twins (DT) in the RGB-D space. Our approach enables efficient real-time DT rendering by semantically describing industrial scenes through object detection and pose estimation of known Computer-Aided Design models. We benchmark tools for real-time, multimodal RGB-D DT creation while tracking consumption of computational resources. Additionally, we provide an extensible and hierarchical annotation strategy for multi-criteria defect detection, unifying pose labelling with logical and structural defect annotations. Based on an automotive use case featuring the quality inspection of an axial flux motor, we demonstrate the effectiveness of our framework. Our results demonstrate detection performace, achieving intersection-over-union (IoU) scores of up to 63.3% compared to ground-truth masks, even if using simple distance measurements under semi-controlled industrial conditions. Our findings lay the groundwork for future research on generalizable, low-data defect detection methods in dynamic manufacturing settings.

Problem

• Zero-shot visual quality inspection for semi-controlled industrial environments
• Real-time 3D Digital Twin simulation to compare with real scenes
• Multi-criteria defect detection with minimal data requirements

Innovation

• Real-time 3D Digital Twin simulation for RGB-D comparison
• Pose-agnostic zero-shot framework using CAD models and object detection
• Extensible hierarchical annotation strategy for multi-criteria defect detection