Fast Rates for Semi-Supervised Learning via Data-Augmentation Graph Regularization

2026-07-08

AI Summary

Existing theory struggles to explain why self-supervised learning achieves high accuracy with very few labeled examples, particularly lacking rigorous guarantees on label efficiency. This work addresses this gap by constructing a similarity graph over unlabeled data via data augmentation and modeling the downstream task as graph Laplacian regularized learning. Leveraging leave-one-out stability analysis, the paper establishes—without relying on idealized assumptions—the first fast transductive error bound of order $O(1/n_L)$ for semi-supervised learning, where $n_L$ denotes the number of labeled samples. The bound explicitly links data augmentation quality to label requirements, revealing that the ability of augmentations to induce graph cuts that cross label boundaries governs performance limits. Formally, the error scales as $C/n_L + R_{\text{DA}}(y)$, where $R_{\text{DA}}(y)$ quantifies the augmentation alignment error, markedly improving upon the $O(1/\sqrt{n_L})$ rate typical of standard supervised learning.

Abstract

Self-supervised learning matches supervised accuracy from a fraction of the labels, but the labeled-sample efficiency behind this has lacked a theoretical explanation. We provide one. Data augmentation induces a similarity graph on the unlabeled data, so downstream learning on that graph is graph-Laplacian-regularized learning. We prove a fast transductive rate, $O(1/n_L)$ in the number of labels, in place of the supervised $O(1/\sqrt{n_L})$, by carrying the leave-one-out stability apparatus of Johnson and Zhang (JMLR 2007) over to the augmentation graph, and without the unrealistic assumptions of limit-based analyses (exact kernel, generalizing features). The bound makes augmentation quality explicit: the expected error is at most $C/n_L + R_{\mathrm{DA}}(y)$, where the data-augmentation alignment error $R_{\mathrm{DA}}(y)$ is the graph-cut mass of augmentations that cross a label boundary, so good augmentations let few labels suffice. The analysis uses a streamlined loss that drops the projector, negative-sample, and orthogonality overhead of standard objectives yet still recovers the top-$K$ ideal features in the infinite-data limit, the augmentation-kernel eigenspace studied by Zhai et al. The result explains the observed accuracy-versus-label-count curve rather than only bounding a generalization gap.

Problem

• semi-supervised learning
• self-supervised learning
• data augmentation
• graph regularization
• label efficiency

Innovation

• data-augmentation graph regularization
• fast transductive rate
• leave-one-out stability
• augmentation alignment error
• semi-supervised learning