FATE: A Prompt-Tuning-Based Semi-Supervised Learning Framework for Extremely Limited Labeled Data

2025-04-14

AI Summary

To address the challenge in extremely low-resource semi-supervised learning—where only 1–5 labeled samples per class are available—where pretrained models struggle to jointly leverage scarce labeled data and abundant unlabeled data, this paper proposes a two-stage “adapt-then-classify” prompt tuning framework. In the first stage, unsupervised prompt tuning and feature alignment adapt the pretrained model to the downstream data distribution. In the second stage, classification is performed using few-shot labeled data, consistency regularization, and pseudo-label refinement. This is the first systematic approach to resolve semi-supervised generalization failure under extreme label sparsity, while unifying support for both vision and vision-language models. Evaluated across seven benchmarks, the method achieves an average improvement of 33.74% over state-of-the-art methods, delivering breakthrough performance particularly in the 1–5 labels-per-class regime.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Large Vision Models
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Semi-supervised learning (SSL) has achieved significant progress by leveraging both labeled data and unlabeled data. Existing SSL methods overlook a common real-world scenario when labeled data is extremely scarce, potentially as limited as a single labeled sample in the dataset. General SSL approaches struggle to train effectively from scratch under such constraints, while methods utilizing pre-trained models often fail to find an optimal balance between leveraging limited labeled data and abundant unlabeled data. To address this challenge, we propose Firstly Adapt, Then catEgorize (FATE), a novel SSL framework tailored for scenarios with extremely limited labeled data. At its core, the two-stage prompt tuning paradigm FATE exploits unlabeled data to compensate for scarce supervision signals, then transfers to downstream tasks. Concretely, FATE first adapts a pre-trained model to the feature distribution of downstream data using volumes of unlabeled samples in an unsupervised manner. It then applies an SSL method specifically designed for pre-trained models to complete the final classification task. FATE is designed to be compatible with both vision and vision-language pre-trained models. Extensive experiments demonstrate that FATE effectively mitigates challenges arising from the scarcity of labeled samples in SSL, achieving an average performance improvement of 33.74% across seven benchmarks compared to state-of-the-art SSL methods. Code is available at https://anonymous.4open.science/r/Semi-supervised-learning-BA72.

Problem

• Addresses semi-supervised learning with extremely scarce labeled data
• Balances pre-trained model adaptation and downstream task performance
• Improves SSL effectiveness for vision and vision-language models

Innovation

• Two-stage prompt tuning paradigm
• Unsupervised adaptation to feature distribution
• Compatible with vision-language models