Anytime-valid, Bayes-assisted,Prediction-Powered Inference

2025-05-23

AI Summary

This paper addresses sequential inference under streaming data with a small number of labeled and a large number of unlabeled observations, aiming to improve the statistical efficiency of confidence intervals beyond traditional methods relying solely on labeled data. Method: We propose a *stop-anytime, universally valid* confidence sequence framework, extending prediction-powered inference (PPI) to the sequential setting for the first time. It adaptively calibrates prediction uncertainty by integrating machine learning predictions with Bayesian priors, and constructs time-uniformly valid confidence sequences via Ville’s inequality and a mixture approach. Contribution/Results: The core innovation lies in achieving prior-guided adaptive efficiency gains while guaranteeing strict coverage validity in online inference. Experiments on both synthetic and real-world datasets demonstrate that our method significantly narrows confidence interval widths while maintaining stable, nominal coverage—outperforming existing sequential inference approaches.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Sequential Decision Making
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Given a large pool of unlabelled data and a smaller amount of labels, prediction-powered inference (PPI) leverages machine learning predictions to increase the statistical efficiency of standard confidence interval procedures based solely on labelled data, while preserving their fixed-time validity. In this paper, we extend the PPI framework to the sequential setting, where labelled and unlabelled datasets grow over time. Exploiting Ville's inequality and the method of mixtures, we propose prediction-powered confidence sequence procedures that are valid uniformly over time and naturally accommodate prior knowledge on the quality of the predictions to further boost efficiency. We carefully illustrate the design choices behind our method and demonstrate its effectiveness in real and synthetic examples.

Problem

• Extends PPI to sequential data settings
• Ensures anytime-valid confidence sequences
• Incorporates prior knowledge for efficiency

Innovation

• Extends PPI to sequential data settings
• Uses Ville's inequality for time validity
• Incorporates prior knowledge for efficiency