Too Open for Opinion? Embracing Open-Endedness in Large Language Models for Social Simulation

2025-10-13

AI Summary

Existing LLM-based social simulation studies predominantly rely on closed-ended question-answering, sacrificing generative richness for evaluability—systematically neglecting viewpoint diversity, reasoning processes, and emergent议题. Method: We propose “open-ended social simulation,” leveraging LLMs’ free-text generation to directly model the semantic richness and individual heterogeneity of public opinion. Our approach integrates large language models, computational text analysis, and survey methodology into an end-to-end open-generation–analysis pipeline. Contribution/Results: This paradigm significantly enhances measurement validity and methodological flexibility; enables unsupervised discovery of perspectives without a priori stance definitions; mitigates researcher conceptual bias; and calls for a novel evaluation framework tailored to generative diversity. By centering open generation and interpretive rigor, our work advances substantive methodological synergy between NLP and the social sciences.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large Language Models (LLMs) are increasingly used to simulate public opinion and other social phenomena. Most current studies constrain these simulations to multiple-choice or short-answer formats for ease of scoring and comparison, but such closed designs overlook the inherently generative nature of LLMs. In this position paper, we argue that open-endedness, using free-form text that captures topics, viewpoints, and reasoning processes "in" LLMs, is essential for realistic social simulation. Drawing on decades of survey-methodology research and recent advances in NLP, we argue why this open-endedness is valuable in LLM social simulations, showing how it can improve measurement and design, support exploration of unanticipated views, and reduce researcher-imposed directive bias. It also captures expressiveness and individuality, aids in pretesting, and ultimately enhances methodological utility. We call for novel practices and evaluation frameworks that leverage rather than constrain the open-ended generative diversity of LLMs, creating synergies between NLP and social science.

Problem

• Simulating social phenomena with constrained LLM formats
• Addressing limitations of closed designs in opinion simulation
• Proposing open-ended approaches for realistic social simulation

Innovation

• Embraces open-ended free-form text generation
• Leverages LLMs' generative diversity for simulation
• Reduces researcher bias through unconstrained responses