Multi-agent decision making: A Blackwell's informativeness approach

2026-05-07

AI Summary

This work addresses the lack of theoretical guarantees on information efficiency in existing multi-agent collaboration methods for large language models (LLMs), such as voting and debate. It introduces Blackwell’s information ordering into the analysis of multi-LLM decision-making for the first time, establishing a theoretical upper bound on information effectiveness through Bayesian posterior pooling. Building on this foundation, the authors propose a scalable question-answering algorithm that estimates and aggregates the posterior distributions of individual agents. Evaluated across six standard question-answering benchmarks, the method significantly outperforms current state-of-the-art multi-LLM collaboration strategies, demonstrating its efficacy in enhancing both information utilization efficiency and decision accuracy.

Abstract

The rapid development of large language models (LLMs) has motivated research on decision-making in multi-agent systems, where multiple agents collaborate to achieve shared objectives. Existing aggregation approaches, such as voting and debate, are largely ad-hoc and lack formal guarantees regarding the informativeness of the resulting decisions. In this paper, we provide a principled approach to analyse decisions made in the multi-LLM setting using Blackwell's informativeness framework. Within the Blackwell information-structure abstraction, we show that voting and debate induce information structures that are no more informative than the pooled private information of all agents. This result identifies Bayesian pooled posterior maximisation as an information-theoretic upper-bound decision rule under the Blackwell ordering. Motivated by this theoretical analysis, we introduce a practical method for LLM-based question-answering (QA) tasks that estimates each agent's posterior and approximates the pooled posterior using a product-of-posteriors estimator. Extensive experiments on six QA benchmarks demonstrate that our approach outperforms state-of-the-art multi-LLM debate and voting methods.

Problem

• multi-agent decision making
• Blackwell informativeness
• information aggregation
• large language models
• posterior pooling

Innovation

• Blackwell informativeness
• multi-agent decision making
• posterior pooling
• large language models
• information structure