Generalized Distribution-Free Semi-Supervised Learning with Risk Rewrite

2026-07-11

AI Summary

This work addresses limitations in existing semi-supervised learning methods, which either rely on strong distributional assumptions or are confined to binary classification with unclear variance optimality. The paper proposes the first distribution-free generalized semi-supervised learning framework for multi-class settings, constructing an unbiased risk estimator through a linear combination of component risks. It extends the risk rewriting approach—previously limited to binary classification—to the multi-class scenario for the first time. Theoretical analysis derives the achievable minimum variance, demonstrating superiority over the PNU method under asymmetric losses, and establishes a direct link between variance reduction and improved generalization performance. Empirical evaluations show that the two proposed algorithms match or outperform current state-of-the-art methods on both binary and multi-class benchmarks.

Abstract

Typical semi-supervised learning (SSL) methods rely on distributional assumptions, and their performance degrades when these are violated. While PNU learning, a risk rewriting method, offers a distribution-free alternative, it is restricted to binary classification and its variance optimality remains unclear. In this paper, we propose a generalized framework that constructs unbiased risk estimators using linear combinations of component risks, subsuming PNU learning and extending to multiclass classification. We derive the minimum achievable variance, demonstrating our estimator can attain lower variance than PNU in asymmetric loss scenarios. Furthermore, we establish a generalization bound directly linking this variance reduction to improved learning performance. Based on these theoretical insights, we introduce two practical SSL methods that empirically match or outperform existing approaches on binary and multiclass benchmarks.

Problem

• semi-supervised learning
• distribution-free
• risk estimation
• multiclass classification
• variance optimality

Innovation

• distribution-free
• semi-supervised learning
• risk rewriting
• variance reduction
• multiclass classification