Feedback-driven object detection and iterative model improvement

2024-11-29

arXiv.org

AI Summary

To address the low efficiency and high cost of manual annotation in object detection, this paper proposes a lightweight semi-automatic annotation framework. The framework integrates an interactive UI, fine-tuned YOLO/RetinaNet models, feedback-driven incremental learning, and versioned annotation snapshot management, enabling image upload, model-assisted labeling, user verification and correction, and iterative refinement. Through quantitative experiments, we demonstrate that—while maintaining or even surpassing human-level annotation accuracy (occasionally exceeding manual precision)—our approach reduces annotation time by up to 53% compared to fully manual annotation, significantly lowering labor and interaction overhead. The platform is open-sourced, accompanied by pedagogical resources and a validated use case on microbial imagery, confirming both workflow efficacy and experimental reproducibility.

Technology Category

• Computer Vision: Object Detection & Categorization
• Machine Learning: Auto ML and Hyperparameter Tuning
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web

Abstract

Automated object detection has become increasingly valuable across diverse applications, yet efficient, high-quality annotation remains a persistent challenge. In this paper, we present the development and evaluation of a platform designed to interactively improve object detection models. The platform allows uploading and annotating images as well as fine-tuning object detection models. Users can then manually review and refine annotations, further creating improved snapshots that are used for automatic object detection on subsequent image uploads - a process we refer to as semi-automatic annotation resulting in a significant gain in annotation efficiency. Whereas iterative refinement of model results to speed up annotation has become common practice, we are the first to quantitatively evaluate its benefits with respect to time, effort, and interaction savings. Our experimental results show clear evidence for a significant time reduction of up to 53% for semi-automatic compared to manual annotation. Importantly, these efficiency gains did not compromise annotation quality, while matching or occasionally even exceeding the accuracy of manual annotations. These findings demonstrate the potential of our lightweight annotation platform for creating high-quality object detection datasets and provide best practices to guide future development of annotation platforms. The platform is open-source, with the frontend and backend repositories available on GitHub (https://github.com/ml-lab-htw/iterative-annotate). To support the understanding of our labeling process, we have created an explanatory video demonstrating the methodology using microscopy images of E. coli bacteria as an example. The video is available on YouTube (https://www.youtube.com/watch?v=CM9uhE8NN5E).

Problem

• Object Detection
• Annotation Efficiency
• Model Optimization

Innovation

• Semi-Automatic Annotation
• Efficiency Improvement
• Object Detection Datasets