Label-Efficient Dataset Pruning via Semi-Supervised Pseudo-Labeling

2026-05-21

AI Summary

This work addresses the limitation of existing dataset pruning methods, which typically rely on fully annotated data and thus struggle in real-world scenarios where labeling is costly yet unlabeled data are abundant. To overcome this, the authors propose SemiPrune, a novel framework that introduces semi-supervised learning into dataset pruning for the first time. Requiring only a small set of randomly labeled samples, SemiPrune leverages pseudo-labeling and dynamic training analysis to accurately estimate sample difficulty and select an informative core subset, without depending on features from pre-trained models. The method achieves state-of-the-art performance on domain-specific, corrupted, and long-tailed datasets, while also demonstrating strong results on standard benchmarks.

Abstract

Dataset pruning reduces the storage and training costs of deep learning by selecting an informative subset from a large dataset. However, most existing pruning methods require fully labeled data, which limits their applicability in realistic settings where unlabeled data are abundant and annotation is costly. Recent label-free pruning methods address this issue, but they rely on features from pretrained models to estimate example difficulty. This dependence can be unreliable when the target dataset differs substantially from the pretraining distribution. We propose SemiPrune, a label-efficient dataset pruning framework, using only a small randomly labeled subset, that uses semi-supervised learning to generate pseudo-labels for unlabeled data, allowing existing supervised pruning methods that require label information to be seamlessly applied to the resulting pseudo-labeled training pool. We then estimate example difficulty from pseudo-label-induced training dynamics and select a coreset. By learning directly from the target dataset, our method better captures the target distribution and provides more reliable signals for difficulty estimation and coreset selection. We validate our approach on domain-specific, image-corrupted, and long-tailed datasets, where it achieves state-of-the-art performance among label-free and label-efficient baselines, while also demonstrating competitive performance on standard benchmarks.

Problem

• dataset pruning
• label efficiency
• semi-supervised learning
• pseudo-labeling
• example difficulty

Innovation

• dataset pruning
• semi-supervised learning
• pseudo-labeling
• label efficiency
• coreset selection