Multiple-Prediction-Powered Inference

2026-03-28

AI Summary

This work addresses the fundamental challenge of efficiently integrating high-cost, high-quality data with multiple low-cost, low-quality proxy sources for statistical inference under a limited budget. The paper introduces MultiPPI, a novel framework that achieves optimal adaptive fusion of heterogeneous predictions from diverse data sources. By explicitly modeling the cost and correlation structure across sources, MultiPPI dynamically allocates resources to construct a statistically efficient estimator that simultaneously attains minimax optimality, finite-sample guarantees, and asymptotic normality. Grounded in statistical estimation theory and optimization techniques, the method significantly reduces estimation error and outperforms existing baselines across three large language model evaluation tasks.

Abstract

Statistical estimation often involves tradeoffs between expensive, high-quality measurements and a variety of lower-quality proxies. We introduce Multiple-Prediction-Powered Inference (MultiPPI): a general framework for constructing statistically efficient estimates by optimally allocating resources across these diverse data sources. This work provides theoretical guarantees about the minimax optimality, finite-sample performance, and asymptotic normality of the MultiPPI estimator. Through experiments across three diverse large language model (LLM) evaluation scenarios, we show that MultiPPI consistently achieves lower estimation error than existing baselines. This advantage stems from its budget-adaptive allocation strategy, which strategically combines subsets of models by learning their complex cost and correlation structures.

Problem

• statistical estimation
• data fusion
• resource allocation
• proxy measurements
• minimax optimality

Innovation

• Multiple-Prediction-Powered Inference
• resource allocation
• statistical efficiency
• budget-adaptive strategy
• LLM evaluation