🤖 AI Summary
This work investigates the generalization capability of transductive learning, focusing on the impact of loss function types, training/test data sizes, and adaptive optimization algorithms. Methodologically, we introduce the novel concept of “transductive hypersamples” to bridge inductive information-theoretic bounds to the transductive setting; derive tight, unified generalization upper bounds—applicable to both random sampling and random partitioning—via mutual information and PAC-Bayes frameworks; and establish new PAC-Bayes bounds under weaker loss and data-size assumptions, extending them to adaptive optimization, semi-supervised learning, and graph learning. Our theoretical analysis reveals that the mutual information between selected training labels and the learned hypothesis critically governs generalization error. Empirically, the derived bounds are validated on synthetic and real-world datasets, demonstrating significant improvements in label efficiency and model robustness.
📝 Abstract
We develop generalization bounds for transductive learning algorithms in the context of information theory and PAC-Bayesian theory, covering both the random sampling setting and the random splitting setting. We show that the transductive generalization gap can be bounded by the mutual information between training labels selection and the hypothesis. By introducing the concept of transductive supersamples, we translate results depicted by various information measures from the inductive learning setting to the transductive learning setting. We further establish PAC-Bayesian bounds with weaker assumptions on the loss function and numbers of training and test data points. Finally, we present the upper bounds for adaptive optimization algorithms and demonstrate the applications of results on semi-supervised learning and graph learning scenarios. Our theoretic results are validated on both synthetic and real-world datasets.