Co-Training with Active Contrastive Learning and Meta-Pseudo-Labeling on 2D Projections for Deep Semi-Supervised Learning

2025-04-25

AI Summary

In image classification scenarios where labeled data are scarce but unlabeled data abundant, conventional semi-supervised learning (SSL) methods heavily rely on pre-trained CNNs, limiting applicability in low-resource domains such as biomedical imaging. Method: We propose active-DeepFA, the first pre-training-free SSL framework for efficient semi-supervised image classification. It integrates supervised contrastive learning, teacher–student meta-pseudo-labeling, and active learning, and innovatively introduces a 2D feature projection–driven cross-network pseudo-label exchange and collaborative annotation strategy to mitigate confirmation bias. Within a co-training architecture, it jointly optimizes supervised contrastive loss, supervised classification loss, and semi-supervised consistency loss. Results: On three biomedical image datasets, active-DeepFA achieves superior performance using only 5% labeled data—outperforming six state-of-the-art methods—and matches their performance with merely 3% labeled data, substantially reducing annotation cost.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Medical and Biological Imaging
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

A major challenge that prevents the training of DL models is the limited availability of accurately labeled data. This shortcoming is highlighted in areas where data annotation becomes a time-consuming and error-prone task. In this regard, SSL tackles this challenge by capitalizing on scarce labeled and abundant unlabeled data; however, SoTA methods typically depend on pre-trained features and large validation sets to learn effective representations for classification tasks. In addition, the reduced set of labeled data is often randomly sampled, neglecting the selection of more informative samples. Here, we present active-DeepFA, a method that effectively combines CL, teacher-student-based meta-pseudo-labeling and AL to train non-pretrained CNN architectures for image classification in scenarios of scarcity of labeled and abundance of unlabeled data. It integrates DeepFA into a co-training setup that implements two cooperative networks to mitigate confirmation bias from pseudo-labels. The method starts with a reduced set of labeled samples by warming up the networks with supervised CL. Afterward and at regular epoch intervals, label propagation is performed on the 2D projections of the networks' deep features. Next, the most reliable pseudo-labels are exchanged between networks in a cross-training fashion, while the most meaningful samples are annotated and added into the labeled set. The networks independently minimize an objective loss function comprising supervised contrastive, supervised and semi-supervised loss components, enhancing the representations towards image classification. Our approach is evaluated on three challenging biological image datasets using only 5% of labeled samples, improving baselines and outperforming six other SoTA methods. In addition, it reduces annotation effort by achieving comparable results to those of its counterparts with only 3% of labeled data.

Problem

• Limited labeled data hinders deep learning model training
• Semi-supervised learning struggles with non-pretrained feature reliance
• Random labeled sampling neglects informative sample selection

Innovation

• Combines contrastive learning and meta-pseudo-labeling
• Uses co-training to mitigate pseudo-label bias
• Integrates active learning for informative sample selection