Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation

2026-04-24

AI Summary

This work addresses the limited generalization of open-vocabulary object detection and the high cost of fully supervised annotation in general-purpose service robots by proposing a semi-supervised label propagation approach for household object segmentation. The method integrates a class-agnostic segmentation proposer with embedding spaces from multiple foundation models—CLIP, ViT, and Theia—and introduces, for the first time, a Hopfield network ensemble to learn cross-modal representative features and enable efficient label propagation. Evaluated in RoboCup@Home scenarios, the approach achieves automatic annotation of 60% of the data using only 40% manually labeled samples, substantially reducing annotation overhead while supporting scalable and efficient labeling across 50 object categories.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Reliable object perception is necessary for general-purpose service robots. Open-vocabulary detectors struggle to generalize beyond a few classes and fully supervised training of object detectors requires time-intensive annotations. We present a semi-supervised label propagation approach for household object segmentation. A segment proposer generates class-agnostic masks, and an ensemble of Hopfield networks assigns labels by learning representative embeddings in complementary foundation model embedding spaces (CLIP, ViT, Theia). Our approach scales to 50 object classes with limited annotation overhead and can automatically label 60% of the data in a RoboCup@Home setting, where preparation time is severely constrained. Dataset and code are publicly available at https://github.com/ais-bonn/label_propagation.

Problem

• image annotation
• object segmentation
• semi-supervised learning
• label propagation
• service robots

Innovation

• semi-supervised learning
• label propagation
• object segmentation
• foundation models
• Hopfield networks