🤖 AI Summary
This study addresses performance bottlenecks of large language models (LLMs) in tasks requiring indirect reasoning, cultural knowledge, and collaborative hypothesis testing by systematically evaluating multi-agent LLM teams in a “What? Where? When?” quiz game. The authors construct teams comprising six open-source models and implement three interaction protocols—voting, silent coordination, and verbose discussion—evaluated on 572 questions from the 2025 Chinese General Knowledge (ChGK) dataset. Results demonstrate that explanatory communication significantly mitigates performance degradation caused by model disagreement, and that a team-leader mechanism exhibits no self-preference bias. The optimal collaborative strategy improves accuracy by 20 percentage points, achieving 44.23%—approaching human team performance—and validates the efficacy of multi-agent interaction in answer selection and error filtering.
📝 Abstract
Large language models (LLMs) remain limited on tasks requiring indirect reasoning, cultural knowledge, and coordinated hypothesis testing. We investigate whether team-based interaction improves LLM performance in What? Where? When? (ChGK), a quiz game designed to reward collective reasoning. We introduce three team strategies: Voting, Silent Team (the captain observes final answers), and Talkative Team (the captain observes both answers and rationales). To minimize data leakage, we evaluate these strategies on a dataset consisting of 572 ChGK questions released in 2025. Using six recent large-scale open models, we show that team-based strategies outperform single-model baselines, yielding gains of up to 20 percentage points in accuracy. The best team achieves 44.23% accuracy, and approaches human team performance on questions with available human statistics. Analysis of inter-model diversity reveals that disagreement strongly predicts lower accuracy, but explanatory communication substantially mitigates performance drops. We further examine captain behavior and find no evidence of self-preference bias; access to peer rationales improves captain judgments. Overall, LLM teams function primarily as answer selection and error-filtering mechanisms rather than generators of novel solutions. Our findings highlight the importance of interaction and suggest adaptive strategies as a promising direction for multi-agent systems.