Plurals: A System for Guiding LLMs Via Simulated Social Ensembles

2024-09-25

arXiv.org

AI Summary

To address inherent viewpoint biases in large language models (LLMs), this paper proposes a pluralistic perspective collaborative reasoning framework grounded in deliberative democracy theory. Methodologically, it constructs configurable identity agents—incorporating nationally representative demographic modeling—orchestrates structured multi-round deliberation, implements dynamic information sharing, and introduces a moderator agent to regulate deliberative quality. Its key contribution lies in the systematic integration of political philosophy’s deliberative democracy paradigm into LLM-based multi-agent system design, achieving theory-driven architectural innovation without compromising engineering scalability. Empirical evaluation demonstrates: (1) six case studies confirm theoretical fidelity; (2) three randomized experiments show that simulated focus group outputs outperform zero-shot baselines on 75% of tasks and exhibit strong alignment with real audience responses (Pearson’s *r* > 0.82).

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Agreement, Argumentation & Negotiation
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Recent debates raised concerns that language models may favor certain viewpoints. But what if the solution is not to aim for a 'view from nowhere' but rather to leverage different viewpoints? We introduce Plurals, a system and Python library for pluralistic AI deliberation. Plurals consists of Agents (LLMs, optionally with personas) which deliberate within customizable Structures, with Moderators overseeing deliberation. Plurals is a generator of simulated social ensembles. Plurals integrates with government datasets to create nationally representative personas, includes deliberation templates inspired by deliberative democracy, and allows users to customize both information-sharing structures and deliberation behavior within Structures. Six case studies demonstrate fidelity to theoretical constructs and efficacy. Three randomized experiments show simulated focus groups produced output resonant with an online sample of the relevant audiences (chosen over zero-shot generation in 75% of trials). Plurals is both a paradigm and a concrete system for pluralistic AI. The Plurals library is available at https://github.com/josh-ashkinaze/plurals and will be continually updated.

Problem

• Addressing bias in LLMs by leveraging diverse viewpoints
• Creating pluralistic AI deliberation with customizable structures
• Ensuring AI output resonates with diverse audience perspectives

Innovation

• Simulated social ensembles guide LLMs
• Nationally representative personas from datasets
• Customizable deliberation structures and behaviors