SOTOPIA-S4: a user-friendly system for flexible, customizable, and large-scale social simulation

2025-04-19

AI Summary

Existing social simulation frameworks lack support for multi-turn, multi-party interactions among LLM-based agents and fail to provide customizable evaluation metrics for rigorous social hypothesis testing. To address these limitations, we propose the first integrated, open-source LLM-powered social simulation system designed specifically for social science research. Our system unifies a configurable simulation engine, modular RESTful APIs, and a lightweight web interface—enabling no-code modeling, execution, and analysis. It adopts a multi-agent LLM architecture with plug-and-play evaluation metrics, ensuring flexibility, scalability, and cross-user accessibility. We validate the system on two real-world scenarios: dyadic recruitment negotiation and multi-party collaborative planning. Results demonstrate substantial improvements in simulation efficiency, customizability of hypothesis validation, and collaborative capability between technical and non-technical users.

Technology Category

• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Social simulation through large language model (LLM) agents is a promising approach to explore and validate hypotheses related to social science questions and LLM agents behavior. We present SOTOPIA-S4, a fast, flexible, and scalable social simulation system that addresses the technical barriers of current frameworks while enabling practitioners to generate multi-turn and multi-party LLM-based interactions with customizable evaluation metrics for hypothesis testing. SOTOPIA-S4 comes as a pip package that contains a simulation engine, an API server with flexible RESTful APIs for simulation management, and a web interface that enables both technical and non-technical users to design, run, and analyze simulations without programming. We demonstrate the usefulness of SOTOPIA-S4 with two use cases involving dyadic hiring negotiation and multi-party planning scenarios.

Problem

• Enables large-scale customizable social simulations with LLM agents
• Reduces technical barriers for multi-party LLM-based interactions
• Provides user-friendly tools for designing and analyzing simulations

Innovation

• Large-scale LLM-based social simulation system
• Customizable multi-turn multi-party interactions
• User-friendly web interface and RESTful APIs