Diversity is the Strength of the AI Crowd

2026-06-28

AI Summary

This study investigates how to enhance the accuracy of forecasting future events under a fixed sampling budget by ensembling predictions from off-the-shelf large language models (LLMs). Building on ensemble learning principles, the approach evaluates multiple LLMs on binary forecasting questions from the Metaculus AI benchmark, highlighting the critical role of error complementarity and predictive diversity among constituent models. The findings demonstrate that models exhibiting low inter-model correlation—such as Grok 4—substantially improve ensemble performance. Moreover, diverse ensembles consistently outperform those composed of homogeneous high-accuracy models, underscoring the pivotal contribution of diversity to effective LLM-based forecasting.

Abstract

Top AI forecasting systems are approaching superforecaster-level accuracy on future world events, but still rely primarily on off-the-shelf LLMs combined with forecasting-specific context gathering and scaffolding. We study how to improve this recipe through ensembling: given a fixed number of samples, which off-the-shelf model forecasts should be combined to maximize accuracy? On binary questions from the Metaculus AI Benchmark, we find that individual accuracy is not enough: many frontier LLMs make highly correlated predictions, limiting the value of additional forecasts from the same or similar models. Instead, the strongest ensembles combine accurate but diverse forecasters, with models such as \model{Grok 4} contributing disproportionately because their predictions are less correlated with other frontier LLMs. These results suggest that the strength of the AI crowd comes not from sampling more forecasts indiscriminately, but from combining forecasts across models with complementary errors, motivating forecasting systems that explicitly optimize for both model quality and diversity.

Problem

• ensembling
• forecasting
• diversity
• large language models
• correlation

Innovation

• ensemble forecasting
• model diversity
• LLM correlation
• complementary errors
• AI forecasting