Bringing Everyone to the Table: An Experimental Study of LLM-Facilitated Group Decision Making

2025-08-11

AI Summary

Unequal information sharing undermines decision quality in group settings, yet the potential of large language models (LLMs) as group facilitators remains insufficiently validated. This study conducts a pre-registered randomized controlled experiment using a five-person hidden-profile task to compare four conditions: no intervention, prompt-based guidance, human facilitation, and GPT-4o facilitation—constituting the first empirical test of LLM-mediated group facilitation. Results show that GPT-4o significantly increases information contributions from the least participative members, thereby mitigating information-sharing inequality; overall information sharing improves, but final decision accuracy does not increase significantly—revealing that enhanced information disclosure alone cannot overcome the hidden-profile effect. Participant attitudes remain unaffected. The study open-sources GRAIL, a reproducible experimental platform for collaborative intelligence, providing both methodological rigor and empirical grounding for future research on AI-augmented group decision-making.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Group decision-making often suffers from uneven information sharing, hindering decision quality. While large language models (LLMs) have been widely studied as aids for individuals, their potential to support groups of users, potentially as facilitators, is relatively underexplored. We present a pre-registered randomized experiment with 1,475 participants assigned to 281 five-person groups completing a hidden profile task--selecting an optimal city for a hypothetical sporting event--under one of four facilitation conditions: no facilitation, a one-time message prompting information sharing, a human facilitator, or an LLM (GPT-4o) facilitator. We find that LLM facilitation increases information shared within a discussion by raising the minimum level of engagement with the task among group members, and that these gains come at limited cost in terms of participants' attitudes towards the task, their group, or their facilitator. Whether by human or AI, there is no significant effect of facilitation on the final decision outcome, suggesting that even substantial but partial increases in information sharing are insufficient to overcome the hidden profile effect studied. To support further research into how LLM-based interfaces can support the future of collaborative decision making, we release our experimental platform, the Group-AI Interaction Laboratory (GRAIL), as an open-source tool.

Problem

• Uneven information sharing in group decision-making
• LLMs' underexplored role as group facilitators
• Effectiveness of LLM facilitation on decision outcomes

Innovation

• LLM facilitation boosts information sharing
• GPT-4o raises minimum engagement levels
• Open-source GRAIL platform supports collaboration