Anomaly triplet-net: progress recognition model using deep metric learning considering occlusion for manual assembly work

2024-11-11

Advanced Robotics

AI Summary

To address the challenge of assembly progress recognition under frequent part occlusion in manual factory assembly, this paper proposes a deep metric learning–based approach. First, YOLO-series models localize the product and crop key regions; then, an Anomaly Triplet-Net is constructed, where anomalous samples—synthetically generated to mimic occlusion-induced distortions—are incorporated into the triplet loss, substantially enhancing robustness to local geometric deformations and appearance variations. This work pioneers the systematic application of an improved metric learning paradigm to occlusion-prone industrial assembly progress estimation. Evaluated on real production-line data, the method achieves 82.9% accuracy in assembly step identification, validating the efficacy and industrial deployability of the end-to-end “detection–cropping–metric recognition” pipeline.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In this paper, a progress recognition method consider occlusion using deep metric learning is proposed to visualize the product assembly process in a factory. First, the target assembly product is detected from images acquired from a fixed-point camera installed in the factory using a deep learning-based object detection method. Next, the detection area is cropped from the image. Finally, by using a classification method based on deep metric learning on the cropped image, the progress of the product assembly work is estimated as a rough progress step. As a specific progress estimation model, we propose an Anomaly Triplet-Net that adds anomaly samples to Triplet Loss for progress estimation considering occlusion. In experiments, an 82.9% success rate is achieved for the progress estimation method using Anomaly Triplet-Net. We also experimented with the practicality of the sequence of detection, cropping, and progression estimation, and confirmed the effectiveness of the overall system.

Problem

• Object Occlusion
• Assembly Progress Prediction
• Manufacturing Industry

Innovation

• Abnormal Triplet Network
• Depth Metric Learning
• Occlusion Robustness