Integrating Semi-Supervised and Active Learning for Semantic Segmentation

2025-01-31

AI Summary

To address the high annotation cost and underutilization of unlabeled data in semantic segmentation, this paper proposes a synergistic framework integrating active learning with enhanced semi-supervised learning. The method jointly leverages prediction-uncertainty-driven hard-sample selection and a cluster-assumption-based pseudo-label automatic refinement (PLAR) mechanism—enabling priority human annotation of challenging samples and dynamic correction of erroneous pseudo-labels. PLAR operates at the pixel level, modeling per-pixel confidence and autonomously refining low-confidence predictions; human annotation is triggered only for samples indeterminate under PLAR. Evaluated on Cityscapes and LoveDA benchmarks, the approach achieves state-of-the-art mIoU improvements of 3.2–4.7 percentage points under identical annotation budgets, demonstrating significantly enhanced model generalization with limited labeled data.

Technology Category

• Machine Learning: Active Learning
• Computer Vision: Segmentation
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we propose a novel active learning approach integrated with an improved semi-supervised learning framework to reduce the cost of manual annotation and enhance model performance. Our proposed approach effectively leverages both the labelled data selected through active learning and the unlabelled data excluded from the selection process. The proposed active learning approach pinpoints areas where the pseudo-labels are likely to be inaccurate. Then, an automatic and efficient pseudo-label auto-refinement (PLAR) module is proposed to correct pixels with potentially erroneous pseudo-labels by comparing their feature representations with those of labelled regions. This approach operates without increasing the labelling budget and is based on the cluster assumption, which states that pixels belonging to the same class should exhibit similar representations in feature space. Furthermore, manual labelling is only applied to the most difficult and uncertain areas in unlabelled data, where insufficient information prevents the PLAR module from making a decision. We evaluated the proposed hybrid semi-supervised active learning framework on two benchmark datasets, one from natural and the other from remote sensing imagery domains. In both cases, it outperformed state-of-the-art methods in the semantic segmentation task.

Problem

• Semantic Segmentation
• Limited Annotated Data
• Unsupervised Learning

Innovation

• Pseudo Label Automatic Refinement
• Semi-supervised Active Learning
• Object Differentiation