The Indispensable Role of User Simulation in the Pursuit of AGI

2025-09-23

AI Summary

Current AGI development faces two fundamental bottlenecks: the difficulty of rigorously evaluating complex interactive systems, and the scarcity of high-quality interactive training data. This paper posits user simulation as the pivotal pathway to overcoming these challenges, introducing a multidisciplinary, large-language-model-based user simulator that achieves high-fidelity modeling of human behavior, interaction patterns, and task objectives. Methodologically, we embed user simulation into the AGI development closed loop to enable scalable automated evaluation, synthetic interactive data generation, and adaptive policy learning. Our core contributions are threefold: (1) the first systematic articulation of user simulation’s foundational role in AGI; (2) the identification of three key technical challenges—behavioral generalizability, interaction-level causality, and verifiable simulation fidelity; and (3) the proposal of a novel “user-simulator–agent co-evolution” research paradigm, establishing a practical, deployable infrastructure for AGI evaluation and training.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior
• Humans and AI: Intelligent User Interfaces

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Progress toward Artificial General Intelligence (AGI) faces significant bottlenecks, particularly in rigorously evaluating complex interactive systems and acquiring the vast interaction data needed for training adaptive agents. This paper posits that user simulation -- creating computational agents that mimic human interaction with AI systems -- is not merely a useful tool, but is a critical catalyst required to overcome these bottlenecks and accelerate AGI development. We argue that realistic simulators provide the necessary environments for scalable evaluation, data generation for interactive learning, and fostering the adaptive capabilities central to AGI. Therefore, research into user simulation technology and intelligent task agents are deeply synergistic and must advance hand-in-hand. This article elaborates on the critical role of user simulation for AGI, explores the interdisciplinary nature of building realistic simulators, identifies key challenges including those posed by large language models, and proposes a future research agenda.

Problem

• Evaluating complex interactive systems rigorously for AGI progress
• Acquiring vast interaction data needed for training adaptive agents
• Developing realistic user simulators to overcome AGI development bottlenecks

Innovation

• User simulation enables scalable evaluation of AGI systems
• Simulators generate interaction data for adaptive learning
• Advancing user simulation and intelligent agents synergistically