🤖 AI Summary
Deep visual models heavily rely on large-scale annotated data, hindering their deployment in low-label-resource scenarios.
Method: This paper presents a unified survey of pseudo-labeling techniques across semi-supervised, self-supervised, and unsupervised learning. We propose, for the first time, a cross-paradigm pseudo-labeling conceptual framework that identifies methodological commonalities—namely label generation, confidence-based filtering, consistency regularization, and dynamic weighting—across these paradigms. We further introduce curriculum learning strategies and self-supervised regularization mechanisms to enable synergistic optimization among paradigms.
Contribution/Results: We establish a comprehensive taxonomy covering pseudo-label generation, filtering, regularization, and cross-paradigm transfer; clarify the technical evolution trajectory; and empirically validate the feasibility of cross-paradigm pseudo-label transfer. Our work provides both theoretical foundations and reproducible practical paradigms for developing vision models with minimal annotation cost.
📝 Abstract
Deep neural models have achieved state of the art performance on a wide range of problems in computer science, especially in computer vision. However, deep neural networks often require large datasets of labeled samples to generalize effectively, and an important area of active research is semi-supervised learning, which attempts to instead utilize large quantities of (easily acquired) unlabeled samples. One family of methods in this space is pseudo-labeling, a class of algorithms that use model outputs to assign labels to unlabeled samples which are then used as labeled samples during training. Such assigned labels, called pseudo-labels, are most commonly associated with the field of semi-supervised learning. In this work we explore a broader interpretation of pseudo-labels within both self-supervised and unsupervised methods. By drawing the connection between these areas we identify new directions when advancements in one area would likely benefit others, such as curriculum learning and self-supervised regularization.