Wisdom Of The (AI) Crowd: Investigating Artificial Swarm Intelligence In Large Language Models

2026-06-30

AI Summary

This study investigates whether large language models (LLMs) can overcome scalability constraints related to cost, coordination, and time by emulating human collective intelligence. We construct an LLM ensemble comprising GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4.5 and systematically evaluate intra-model sampling and inter-model aggregation strategies across eight estimation tasks. Our work provides the first empirical evidence that LLMs exhibit collective intelligence–like error reduction capabilities and demonstrate metacognitive awareness in uncertainty assessment. Results show that both aggregation approaches significantly reduce estimation errors—by up to 37 percentage points in mean absolute percentage error (MAPE)—and reveal a strong positive correlation between confidence interval width and estimation error (ρ = 0.242–0.568, p < 0.001).

Abstract

Human swarm intelligence demonstrates remarkable collective accuracy but faces scalability constraints in cost, coordination, and time. We investigate whether large language models (LLMs) can approximate swarm intelligence effects through artificial swarms, addressing a critical gap in understanding AI-based aggregation mechanisms. We conducted a controlled experiment with 960 manually executed prompts across three proprietary models (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5), testing intra-model sampling and inter-model aggregation on eight estimation tasks. Results reveal consistent error reduction through intra- and inter-model aggregation, with significant error reductions up to 37 percentage points in MAPE across different aggregation strategies. We observed small to large effect sizes for positive correlations (Spearman's $ρ=0.242-0.568$, all $p<0.001$) between relative confidence interval widths and relative estimation errors, suggesting LLMs possess metacognitive awareness when assessing uncertainty. We discuss implications for research and practice, providing actionable insights for deploying LLM swarms in organizational decision-making.

Problem

• artificial swarm intelligence
• large language models
• collective accuracy
• aggregation mechanisms
• scalability constraints

Innovation

• artificial swarm intelligence
• large language models
• intra-model aggregation
• inter-model aggregation
• metacognitive awareness