Hybrid Synthetic Data Generation with Domain Randomization Enables Zero-Shot Vision-Based Part Inspection Under Extreme Class Imbalance

2025-11-28

AI Summary

Industrial visual inspection faces dual challenges of scarce defective samples and prohibitively high costs for accurate real-world annotations, leading to poor model generalization and deployment difficulties. To address this, we propose a zero-shot visual inspection framework that eliminates reliance on real defective samples by synthesizing a fully annotated hybrid dataset—integrating physics-based rendering, domain randomization, and compositing onto real-world backgrounds. Our method innovatively combines high-fidelity rendering with stochastic textural and illumination variations drawn from real scenes, substantially enhancing cross-domain generalization. Evaluated on a lightweight architecture (YOLOv8n for detection and MobileNetV3-small for classification), the framework achieves 0.995 mAP@0.5, 96% classification accuracy, and 90.1% balanced accuracy across 300 real industrial components—significantly outperforming few-shot baselines (50% improvement).

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Machine learning, particularly deep learning, is transforming industrial quality inspection. Yet, training robust machine learning models typically requires large volumes of high-quality labeled data, which are expensive, time-consuming, and labor-intensive to obtain in manufacturing. Moreover, defective samples are intrinsically rare, leading to severe class imbalance that degrades model performance. These data constraints hinder the widespread adoption of machine learning-based quality inspection methods in real production environments. Synthetic data generation (SDG) offers a promising solution by enabling the creation of large, balanced, and fully annotated datasets in an efficient, cost-effective, and scalable manner. This paper presents a hybrid SDG framework that integrates simulation-based rendering, domain randomization, and real background compositing to enable zero-shot learning for computer vision-based industrial part inspection without manual annotation. The SDG pipeline generates 12,960 labeled images in one hour by varying part geometry, lighting, and surface properties, and then compositing synthetic parts onto real image backgrounds. A two-stage architecture utilizing a YOLOv8n backbone for object detection and MobileNetV3-small for quality classification is trained exclusively on synthetic data and evaluated on 300 real industrial parts. The proposed approach achieves an mAP@0.5 of 0.995 for detection, 96% classification accuracy, and 90.1% balanced accuracy. Comparative evaluation against few-shot real-data baseline approaches demonstrates significant improvement. The proposed SDG-based approach achieves 90-91% balanced accuracy under severe class imbalance, while the baselines reach only 50% accuracy. These results demonstrate that the proposed method enables annotation-free, scalable, and robust quality inspection for real-world manufacturing applications.

Problem

• Generates synthetic data for zero-shot part inspection
• Addresses class imbalance in defect detection training
• Enables robust quality inspection without manual annotation

Innovation

• Hybrid synthetic data generation with domain randomization
• Zero-shot learning for vision-based part inspection
• Two-stage YOLOv8 and MobileNetV3 architecture