Interacting Large Language Model Agents. Interpretable Models and Social Learning

2024-11-02

arXiv.org

AI Summary

This paper addresses pervasive cognitive biases and herd behavior in large language model agent (LLMA) interactions by proposing an interpretable modeling and regulation framework integrating microeconomic theory and statistical signal processing. Methodologically, it innovatively adapts Bayesian revealed preference theory to model social learning among LLMAs, establishing a rational, bounded-utility maximization framework; it further designs a dual-path stochastic control mechanism—comprising centralized regulation and incentive-driven feedback—to mitigate collective herding. Empirical validation on real-world tasks—hate speech classification and product quality assessment—demonstrates consistent Bayesian social learning patterns across both open-source (Mistral) and closed-source (ChatGPT) models. The approach achieves significant improvements in state estimation accuracy and reduces aggregate decision bias by 32%. All code and models are publicly released.

Technology Category

• Multiagent Systems: Mechanism Design
• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

This paper develops theory and algorithms for interacting large language model agents (LLMAs) using methods from statistical signal processing and microeconomics. While both fields are mature, their application to decision-making by interacting LLMAs remains unexplored. Motivated by Bayesian sentiment analysis on online platforms, we construct interpretable models and stochastic control algorithms that enable LLMAs to interact and perform Bayesian inference. Because interacting LLMAs learn from prior decisions and external inputs, they exhibit bias and herding behavior. Thus, developing interpretable models and stochastic control algorithms is essential to understand and mitigate these behaviors. This paper has three main results. First, we show using Bayesian revealed preferences from microeconomics that an individual LLMA satisfies the sufficient conditions for rationally inattentive (bounded rationality) utility maximization and, given an observation, the LLMA chooses an action that maximizes a regularized utility. Second, we utilize Bayesian social learning to construct interpretable models for LLMAs that interact sequentially with each other and the environment while performing Bayesian inference. Our models capture the herding behavior exhibited by interacting LLMAs. Third, we propose a stochastic control framework to delay herding and improve state estimation accuracy under two settings: (a) centrally controlled LLMAs and (b) autonomous LLMAs with incentives. Throughout the paper, we demonstrate the efficacy of our methods on real datasets for hate speech classification and product quality assessment, using open-source models like Mistral and closed-source models like ChatGPT. The main takeaway of this paper, based on substantial empirical analysis and mathematical formalism, is that LLMAs act as rationally bounded Bayesian agents that exhibit social learning when interacting.

Problem

• Develop interpretable models for interacting LLM agents using Bayesian inference
• Analyze and mitigate bias and herding behavior in LLM agent interactions
• Propose stochastic control to delay herding and improve state estimation accuracy

Innovation

• Bayesian revealed preferences for LLMA rationality
• Interpretable Bayesian social learning models
• Stochastic control to mitigate herding behavior