Robust Assembly Progress Estimation via Deep Metric Learning

2026-01-01

arXiv.org

AI Summary

This work addresses the challenge of inaccurate assembly progress estimation in multi-day manual assembly tasks, where subtle visual variations or occlusions often lead to misclassification and reduced efficiency in smart factories. To mitigate errors caused by high visual similarity between adjacent assembly stages, the authors propose Anomaly Quadruplet-Net, a deep metric learning framework that integrates a quadruplet loss function with a targeted sample selection mechanism. By employing a customized data loading strategy and robust visual feature extraction, the method enhances the discriminability of visually similar phases. Evaluated on a desktop PC assembly image dataset, the approach improves progress estimation accuracy by 1.3% and reduces misclassification rates between consecutive tasks by 1.9%, demonstrating significantly improved robustness in assembly progress recognition.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

In recent years, the advancement of AI technologies has accelerated the development of smart factories. In particular, the automatic monitoring of product assembly progress is crucial for improving operational efficiency, minimizing the cost of discarded parts, and maximizing factory productivity. However, in cases where assembly tasks are performed manually over multiple days, implementing smart factory systems remains a challenge. Previous work has proposed Anomaly Triplet-Net, which estimates assembly progress by applying deep metric learning to the visual features of products. Nevertheless, when visual changes between consecutive tasks are subtle, misclassification often occurs. To address this issue, this paper proposes a robust system for estimating assembly progress, even in cases of occlusion or minimal visual change, using a small-scale dataset. Our method leverages a Quadruplet Loss-based learning approach for anomaly images and introduces a custom data loader that strategically selects training samples to enhance estimation accuracy. We evaluated our approach using a image datasets: captured during desktop PC assembly. The proposed Anomaly Quadruplet-Net outperformed existing methods on the dataset. Specifically, it improved the estimation accuracy by 1.3% and reduced misclassification between adjacent tasks by 1.9% in the desktop PC dataset and demonstrating the effectiveness of the proposed method.

Problem

• assembly progress estimation
• visual change
• occlusion
• manual assembly
• misclassification

Innovation

• Quadruplet Loss
• Deep Metric Learning
• Assembly Progress Estimation
• Anomaly Detection
• Small-scale Dataset