A Synthetic-Driven Vision System for Assembly Step Recognition

2026-06-30

AI Summary

This work addresses the lack of efficient, low-cost real-time quality inspection methods in industrial assembly, where existing vision-based approaches rely heavily on expensive collection and annotation of real-world data. The authors propose a novel method that leverages only CAD models and step descriptions to automatically generate photorealistic synthetic assembly sequences, enabling the training of a high-accuracy, real-time assembly step recognition model within one hour. The approach innovatively integrates physics-based motion generation to simulate the variability of human assembly actions, domain-randomized rendering to enhance robustness to environmental variations, and an object detection–driven mechanism for step recognition. Evaluated in real industrial settings, the model achieves 92.4% accuracy, outperforming current baselines by 46.7%, 15.8%, and 61.2%, respectively.

Abstract

Quality control in industrial assembly is essential, and real-time monitoring of the assembly process is crucial for preventing costly defects and ensuring production reliability. Vision-based automated inspection offers a powerful solution for such real-time monitoring. However, due to the specialized industrial components and processes, training these models typically relies on task-specific real-world data, which is costly and labor-intensive to collect and annotate. In this paper, we propose a system that automatically generates realistic assembly sequences and further trains real-time inspection models using the synthetic data. It can be efficiently applied to a given task within an hour, requiring only CAD models and simple step descriptions. Focusing on practical challenges, our system integrates a physics-based motion generation module to capture the variance of different human assembly, designs domain-randomized rendering to deal with the environmental complexity and variation, and employs an object-detection-based step recognition module for robust sim-to-real transfer, leading to 92.4% accuracy on a real-world assembly case with 46.7%, 15.8% and 61.2% performance improvement, respectively. Overall, our system provides a practical solution for industrial assembly inspection without requiring expensive real-world data collection and annotation, with the effectiveness validated on real industrial assembly tasks.

Problem

• assembly step recognition
• real-time monitoring
• synthetic data
• industrial quality control
• vision-based inspection

Innovation

• synthetic data
• physics-based motion generation
• domain randomization
• sim-to-real transfer
• assembly step recognition