From Semantic To Instance: A Semi-Self-Supervised Learning Approach

2025-06-19

AI Summary

To address the challenges of dense occlusion and high annotation costs in instance segmentation for agricultural scenarios, this paper proposes a weakly supervised paradigm that requires only a small number of semantic annotations and leverages semantic segmentation to guide the generation of high-quality instance masks. Our method introduces three key contributions: (1) GLMask—a novel image-mask representation that decouples color interference and enhances shape and texture modeling; (2) the first semantic-to-instance semi-supervised conversion framework; and (3) a hybrid architecture integrating Transformer and CNN modules within an end-to-end instance generation pipeline. Evaluated on a wheat ear dataset, our approach achieves 98.5% mAP@50 and improves COCO benchmark performance by 12.6 percentage points. It demonstrates strong robustness across both domain-specific agricultural applications and general-purpose scenes, significantly reducing reliance on costly pixel-level instance annotations.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Instance segmentation is essential for applications such as automated monitoring of plant health, growth, and yield. However, extensive effort is required to create large-scale datasets with pixel-level annotations of each object instance for developing instance segmentation models that restrict the use of deep learning in these areas. This challenge is more significant in images with densely packed, self-occluded objects, which are common in agriculture. To address this challenge, we propose a semi-self-supervised learning approach that requires minimal manual annotation to develop a high-performing instance segmentation model. We design GLMask, an image-mask representation for the model to focus on shape, texture, and pattern while minimizing its dependence on color features. We develop a pipeline to generate semantic segmentation and then transform it into instance-level segmentation. The proposed approach substantially outperforms the conventional instance segmentation models, establishing a state-of-the-art wheat head instance segmentation model with mAP@50 of 98.5%. Additionally, we assessed the proposed methodology on the general-purpose Microsoft COCO dataset, achieving a significant performance improvement of over 12.6% mAP@50. This highlights that the utility of our proposed approach extends beyond precision agriculture and applies to other domains, specifically those with similar data characteristics.

Problem

• Reducing manual annotation for instance segmentation models
• Improving segmentation in dense, self-occluded agricultural images
• Enhancing generalizability across domains with similar data traits

Innovation

• Semi-self-supervised learning with minimal manual annotation
• GLMask focuses on shape, texture, and pattern
• Pipeline converts semantic to instance-level segmentation