Robust Aggregation of Calibrated Forecasts

2026-06-29

AI Summary

This work investigates how to robustly aggregate multiple calibrated but potentially under-informed expert predictions for decision-making, assuming only that the experts satisfy calibration. To this end, it introduces a max-min robust benchmark grounded in calibration constraints, which leverages additional decision-relevant information embedded in the joint distribution of expert forecasts by maximizing worst-case expected utility. This benchmark strictly dominates the classical ex-post optimal benchmark—particularly when accounting for calibration errors—and can be computed efficiently via linear programming. Furthermore, the paper designs an online learning algorithm that asymptotically achieves this robust performance bound using only predictions or state feedback, thereby revealing the added value inherent in the information structure of calibrated expert advice.

Abstract

Decision-makers often rely on multiple probabilistic forecasts that are individually calibrated but need not be fully informative. We develop a framework for aggregating such forecasts when the decision-maker knows only that experts satisfy calibration. We show that the joint distribution of calibrated forecasts can contain decision-relevant information that is unavailable from any single expert, so the standard optimal-in-hindsight (OIH) benchmark may substantially understate attainable performance. To formalize this idea, we introduce a robust max-min benchmark: the best payoff a decision-maker can guarantee against all profile-wise conditional-mean mappings compatible with calibration. This benchmark is tractable, admits a linear-programming formulation, and dominates the OIH benchmark up to calibration error. It can nevertheless be strictly below the Bayesian benchmark, clarifying the value of knowing experts' information structures. Finally, we provide online algorithms that attain the robust benchmark under forecast-only feedback and stronger contextual benchmarks under state feedback.

Problem

• forecast aggregation
• calibration
• decision-making
• probabilistic forecasting
• robust benchmark

Innovation

• robust aggregation
• calibrated forecasts
• max-min benchmark
• online learning
• forecast combination