Leveraging LLM-based agents for social science research: insights from citation network simulations

2025-11-05

AI Summary

This study investigates the boundaries of large language models (LLMs) in social simulation, specifically their capacity to model human scholarly behavior and generate citation networks. Method: We propose two novel paradigms—LLM-SE (LLM-based Social Experiment) and LLM-LE (LLM-based Citation Evolution)—that systematically deploy LLM agents for reproducible, idealized social simulation. Using agent-based modeling, synthetic citation network generation, and power-law distribution analysis, we quantitatively reproduce key empirical phenomena observed in real academic ecosystems: power-law degree distributions, citation distortion, and network diameter contraction. Contribution/Results: Our work empirically validates LLMs as effective tools for computational social science simulation. It establishes a scalable, controllable experimental platform that shifts social simulation from descriptive modeling toward mechanism-driven theoretical testing and extension, thereby advancing rigorous, theory-grounded inquiry into scholarly dynamics.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The emergence of Large Language Models (LLMs) demonstrates their potential to encapsulate the logic and patterns inherent in human behavior simulation by leveraging extensive web data pre-training. However, the boundaries of LLM capabilities in social simulation remain unclear. To further explore the social attributes of LLMs, we introduce the CiteAgent framework, designed to generate citation networks based on human-behavior simulation with LLM-based agents. CiteAgent successfully captures predominant phenomena in real-world citation networks, including power-law distribution, citational distortion, and shrinking diameter. Building on this realistic simulation, we establish two LLM-based research paradigms in social science: LLM-SE (LLM-based Survey Experiment) and LLM-LE (LLM-based Laboratory Experiment). These paradigms facilitate rigorous analyses of citation network phenomena, allowing us to validate and challenge existing theories. Additionally, we extend the research scope of traditional science of science studies through idealized social experiments, with the simulation experiment results providing valuable insights for real-world academic environments. Our work demonstrates the potential of LLMs for advancing science of science research in social science.

Problem

• Exploring LLM capabilities in social behavior simulation
• Developing CiteAgent framework for citation network generation
• Establishing LLM-based research paradigms for social science

Innovation

• Introduces CiteAgent framework for citation network simulation
• Establishes LLM-SE and LLM-LE research paradigms
• Uses LLM-based agents for social science experiments