Algorithmic Expert Aggregation

2026-07-09

AI Summary

This work addresses the problem of aggregating multiple calibrated Bayesian expert forecasts to construct a new predictor that remains calibrated and is Blackwell-dominated by the target expert, rather than merely minimizing a specific loss function. Under the setting where only the experts’ prior distributions are observed—without access to the true state—the authors formally define the aggregation objective as simultaneously achieving calibration and Blackwell refinability. By modeling calibrated experts through reduced-form information structures, they characterize the set of feasible predictions using the row space of a linear system intersected with a non-negative cone, and analyze it via Blackwell dominance theory. Their main contributions include efficient solvability of both randomized aggregation problems, while showing that determining the existence of a deterministic aggregator is NP-hard and admits no multiplicative PTAS unless P = NP, thereby revealing a fundamental computational distinction between randomized and deterministic aggregation.

Abstract

Forecast aggregation aims to combine information from multiple Bayesian experts' forecasts into an aggregate forecast. In much of this literature, however, the aggregate forecast is optimized for a particular loss or robustness criterion and need not itself be calibrated with respect to the outcome. We introduce and study expert aggregation, where the goal is instead to aggregate Bayesian experts into a new expert that continues to provide calibrated forecasts. In particular, we consider a setting where each input expert reports calibrated predictions, and the aggregator observes the prior distribution over states, and the input experts, but not the underlying Bayes probabilities of the states. We ask whether one can (i) construct a calibrated output expert that Blackwell refines a target expert and cannot be further Blackwell improved using the available information; and (ii) when a proper loss is specified, compute a nearly loss-optimal expert among all such refinements. We formulate calibrated experts as reduced-form information structures and measure refinement by Blackwell dominance of the induced prediction distributions. We characterize the constructible output experts through observable linear information: the input experts generate a linear system whose row space determines which calibrated output predictions are identifiable, and a new expert is constructible exactly when its predictions lie in the associated observable nonnegative cone. We establish a sharp algorithmic picture. When randomized output experts are allowed, both questions above admit efficient algorithms. In contrast, deterministic output experts are computationally intractable: deciding whether a deterministic calibrated refinement exists is $\mathsf{NP}$-hard, and deterministic proper-loss optimization admits no multiplicative PTAS unless $\mathsf{P}=\mathsf{NP}$.

Problem

• forecast aggregation
• calibrated forecasts
• Blackwell refinement
• Bayesian experts
• proper loss

Innovation

• calibrated forecasting
• Blackwell dominance
• expert aggregation
• information structures
• computational complexity