Optimal Inference with Black-box Predictions

2026-08-10

AI Summary

This study addresses the problem of effectively integrating black-box predictions with observed data to enhance hypothesis testing performance in high-dimensional Gaussian sequence models. Focusing on settings where prediction accuracy is unknown and multiple predictions may exhibit strong alignment, the work establishes, for the first time, the information-theoretic limits of such auxiliary inference. Building upon these fundamental limits, the authors develop an adaptive, computationally efficient, and statistically powerful testing procedure. The proposed method requires no prior knowledge of prediction quality and automatically leverages structural dependencies among predictions. It rigorously controls Type I error while substantially improving statistical power, with pronounced gains particularly evident when multiple predictions are highly consistent.

Abstract

Powerful black-box predictive models have motivated many proposals for combining observed data with predictions to perform valid statistical inference. Despite this progress, the field lacks a unifying principle that explains how hypothesis tests should integrate data and predictions in a way that is both valid and efficient. In this work, we address this gap in the high-dimensional Gaussian sequence model. We characterize the information-theoretic limits of inference with black-box predictions when their accuracies are known and, for orthogonal predictions, when they are unknown. Building on these characterizations, we develop practical hypothesis tests that adapt to the unknown accuracies of the predictions while benefiting from strong alignment among them.

Problem

• statistical inference
• black-box predictions
• high-dimensional Gaussian sequence model
• hypothesis testing
• prediction accuracy

Innovation

• black-box predictions
• statistical inference
• information-theoretic limits
• adaptive hypothesis testing
• high-dimensional Gaussian model