Information Design With Large Language Models

2025-09-29

AI Summary

This paper addresses the longstanding disconnect between linguistic framing and Bayesian signaling mechanisms in information design. Traditional approaches assume receivers perform strict Bayesian belief updates, ignoring systematic non-Bayesian belief shifts induced by phrasing. We propose the first unified theoretical and computational framework that jointly optimizes framing and signaling. We formalize framing’s systematic influence on belief formation and introduce large language models as differentiable “belief prediction agents” to instantiate a framing-to-belief mapping. Within the language space, we perform end-to-end optimization via gradient-informed local search (hill climbing). Our method transcends the strict Bayesian paradigm, achieving significant improvements in persuasive efficacy across two marketing scenarios. Both automated evaluation and human annotation consistently validate its effectiveness.

Technology Category

• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Multiagent Systems: Mechanism Design

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems

Abstract

Information design is typically studied through the lens of Bayesian signaling, where signals shape beliefs based on their correlation with the true state of the world. However, Behavioral Economics and Psychology emphasize that human decision-making is more complex and can depend on how information is framed. This paper formalizes a language-based notion of framing and bridges this to the popular Bayesian-persuasion model. We model framing as a possibly non-Bayesian, linguistic way to influence a receiver's belief, while a signaling (or recommendation) scheme can further refine this belief in the classic Bayesian way. A key challenge in systematically optimizing in this framework is the vast space of possible framings and the difficulty of predicting their effects on receivers. Based on growing evidence that Large Language Models (LLMs) can effectively serve as proxies for human behavior, we formulate a theoretical model based on access to a framing-to-belief oracle. This model then enables us to precisely characterize when solely optimizing framing or jointly optimizing framing and signaling is tractable. We substantiate our theoretical analysis with an empirical algorithm that leverages LLMs to (1) approximate the framing-to-belief oracle, and (2) optimize over language space using a hill-climbing method. We apply this to two marketing-inspired case studies and validate the effectiveness through analytical and human evaluation.

Problem

• Formalizing language-based framing in information design
• Bridging behavioral framing with Bayesian persuasion models
• Optimizing framing and signaling using LLM proxies

Innovation

• Leveraging LLMs as human behavior proxies
• Combining linguistic framing with Bayesian persuasion
• Optimizing language using hill-climbing algorithms