Diverse Evidence, Better Forecasts: Multi-Agent Deliberation Under Information Asymmetry

2026-07-01

AI Summary

This work addresses the limitation of existing multi-agent forecasting approaches, where information homogenization often induces herding behavior, thereby constraining belief updating and performance gains. To overcome this, the authors propose InfoDelphi, a novel framework that systematically introduces designed information asymmetry by partitioning evidence into shared and mutually exclusive private subsets, endowing each agent with unique knowledge. Effective calibration is achieved through correlation-aware evidence routing, iterative reasoning-based negotiation, and confidence-weighted aggregation. Theoretically, this mechanism reduces inter-agent error correlation, highlighting input diversity as essential for negotiation gains. On the PolyGym benchmark, InfoDelphi improves Brier scores by 12–18% over both the strongest single-agent and multi-agent baselines and increases accuracy by 4–8 percentage points. Ablation studies confirm that information asymmetry is critical to the negotiation efficacy.

Abstract

Multi-agent systems are increasingly used for forecasting future events, as deliberation among multiple LLMs is believed to improve reasoning and calibration. Yet existing approaches overlook a critical design choice: what information each agent receives. When all agents are given identical evidence, deliberation collapses into herding rather than genuine belief revision, leaving multi-agent systems little better than a single agent. We identify this as a fundamental gap and propose designed information asymmetry to close it: by partitioning evidence into shared public and disjoint private subsets, each agent holds exclusive knowledge that can only reach others through deliberation. We theoretically show that this decomposition reduces inter-agent error correlation, and instantiate it in InfoDelphi, a framework combining relevance-aware evidence routing, rationale-based iterative deliberation, and confidence-weighted aggregation. On PolyGym, a benchmark of 375 binary forecasting questions derived from real-world prediction markets, InfoDelphi outperforms the strongest single-agent and multi-agent baselines by 12--18% in Brier score and 4--8 percentage points in accuracy. More detailed experiments confirm that removing information asymmetry eliminates most deliberation gains, establishing diversity of input as the key enabler of effective multi-agent reasoning.

Problem

• multi-agent systems
• forecasting
• information asymmetry
• evidence diversity
• deliberation

Innovation

• information asymmetry
• multi-agent deliberation
• evidence partitioning
• LLM-based forecasting
• InfoDelphi