Physical Annotation for Automated Optical Inspection: A Concept for In-Situ, Pointer-Based Trainingdata Generation

2025-06-05

AI Summary

To address the inefficiency, geometric distortion, and difficulty of directly embedding domain expertise into machine learning pipelines caused by conventional screen-based annotation in automated optical inspection (AOI), this paper proposes a physics-aware in-situ annotation system based on a physical pointer. The system enables quality inspectors to naturally sketch defect trajectories directly on workpiece surfaces, achieving zero-loss mapping from expert knowledge to structured annotation data. Key technical components include high-precision spatial tracking of the pointer, real-time dynamic calibration, projection-based guidance interface, and standardized format conversion compatible with open-source annotation platforms such as CVAT. Experimental evaluation demonstrates millimeter-level trajectory acquisition accuracy, significantly improving annotation consistency and efficiency while eliminating the need for IT expertise. The approach effectively mitigates sample attrition and annotation distortion commonly observed in traditional AOI workflows.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Hardware-aware ML
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper introduces a novel physical annotation system designed to generate training data for automated optical inspection. The system uses pointer-based in-situ interaction to transfer the valuable expertise of trained inspection personnel directly into a machine learning (ML) training pipeline. Unlike conventional screen-based annotation methods, our system captures physical trajectories and contours directly on the object, providing a more intuitive and efficient way to label data. The core technology uses calibrated, tracked pointers to accurately record user input and transform these spatial interactions into standardised annotation formats that are compatible with open-source annotation software. Additionally, a simple projector-based interface projects visual guidance onto the object to assist users during the annotation process, ensuring greater accuracy and consistency. The proposed concept bridges the gap between human expertise and automated data generation, enabling non-IT experts to contribute to the ML training pipeline and preventing the loss of valuable training samples. Preliminary evaluation results confirm the feasibility of capturing detailed annotation trajectories and demonstrate that integration with CVAT streamlines the workflow for subsequent ML tasks. This paper details the system architecture, calibration procedures and interface design, and discusses its potential contribution to future ML data generation for automated optical inspection.

Problem

• Generates training data for automated optical inspection
• Uses pointer-based in-situ interaction for annotation
• Bridges human expertise with machine learning training

Innovation

• Pointer-based in-situ interaction for annotation
• Calibrated tracked pointers to record user input
• Projector interface for visual guidance assistance