Bayesian Semi-supervised Inference via a Debiased Modeling Approach

2025-09-22

AI Summary

This paper addresses the low estimation efficiency of population mean estimation in semi-supervised learning. We propose a novel Bayesian debiased inference method tailored to settings with abundant unlabeled data and a moderate-sized labeled sample. Our approach achieves automatic bias correction via sample splitting, modeling of summary statistics, and flexible learning of the nuisance function. It constitutes the first systematic integration of the Bayesian framework into semi-supervised parametric inference, introducing a debiased-representation-based modeling paradigm that overcomes bias arising from misspecified or slowly converging nuisance parameters in conventional methods. Theoretically, the estimator satisfies the Bernstein–von Mises theorem—ensuring asymptotic normality and achieving the optimal convergence rate. Numerical experiments demonstrate substantial improvements over existing semi-supervised estimators across diverse simulation settings.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Reasoning under Uncertainty: Probabilistic Inference
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Inference in semi-supervised (SS) settings has gained substantial attention in recent years due to increased relevance in modern big-data problems. In a typical SS setting, there is a much larger-sized unlabeled data, containing only observations of predictors, and a moderately sized labeled data containing observations for both an outcome and the set of predictors. Such data naturally arises when the outcome, unlike the predictors, is costly or difficult to obtain. One of the primary statistical objectives in SS settings is to explore whether parameter estimation can be improved by exploiting the unlabeled data. We propose a novel Bayesian method for estimating the population mean in SS settings. The approach yields estimators that are both efficient and optimal for estimation and inference. The method itself has several interesting artifacts. The central idea behind the method is to model certain summary statistics of the data in a targeted manner, rather than the entire raw data itself, along with a novel Bayesian notion of debiasing. Specifying appropriate summary statistics crucially relies on a debiased representation of the population mean that incorporates unlabeled data through a flexible nuisance function while also learning its estimation bias. Combined with careful usage of sample splitting, this debiasing approach mitigates the effect of bias due to slow rates or misspecification of the nuisance parameter from the posterior of the final parameter of interest, ensuring its robustness and efficiency. Concrete theoretical results, via Bernstein--von Mises theorems, are established, validating all claims, and are further supported through extensive numerical studies. To our knowledge, this is possibly the first work on Bayesian inference in SS settings, and its central ideas also apply more broadly to other Bayesian semi-parametric inference problems.

Problem

• Estimating population mean efficiently using both labeled and unlabeled data
• Developing Bayesian debiasing method to mitigate nuisance parameter misspecification effects
• Establishing robust semi-supervised inference via targeted summary statistics modeling

Innovation

• Bayesian semi-supervised method for population mean estimation
• Debiased modeling using summary statistics and nuisance function
• Sample splitting to mitigate bias and ensure robustness