RecoWorld: Building Simulated Environments for Agentic Recommender Systems

2025-09-12

AI Summary

Training recommendation agents in real-user environments incurs negative externalities, such as user dissatisfaction and engagement loss. Method: This paper introduces RecoWorld—a scalable, agent-oriented simulation environment for recommender systems. Its design centers on a dual-perspective architecture and a novel “user-guidance–recommendation-response” paradigm. It integrates large language model–driven reflective instruction generation, user mental state evolution modeling, multimodal content representation, and multi-agent collaborative simulation. Furthermore, it supports multi-turn reinforcement learning optimized for user retention. Contribution/Results: Experiments demonstrate that RecoWorld significantly improves long-term user engagement and system retention rates. As the first dynamic training platform for agent-based recommendation, it uniquely balances realism, controllability, and reproducibility—enabling safe, efficient, and principled agent development without real-world deployment risks.

Technology Category

• Multiagent Systems: Adversarial Agents
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present RecoWorld, a blueprint for building simulated environments tailored to agentic recommender systems. Such environments give agents a proper training space where they can learn from errors without impacting real users. RecoWorld distinguishes itself with a dual-view architecture: a simulated user and an agentic recommender engage in multi-turn interactions aimed at maximizing user retention. The user simulator reviews recommended items, updates its mindset, and when sensing potential user disengagement, generates reflective instructions. The agentic recommender adapts its recommendations by incorporating these user instructions and reasoning traces, creating a dynamic feedback loop that actively engages users. This process leverages the exceptional reasoning capabilities of modern LLMs. We explore diverse content representations within the simulator, including text-based, multimodal, and semantic ID modeling, and discuss how multi-turn RL enables the recommender to refine its strategies through iterative interactions. RecoWorld also supports multi-agent simulations, allowing creators to simulate the responses of targeted user populations. It marks an important first step toward recommender systems where users and agents collaboratively shape personalized information streams. We envision new interaction paradigms where "user instructs, recommender responds," jointly optimizing user retention and engagement.

Problem

• Building simulated environments for agentic recommender systems training
• Creating dual-view architecture for user-recommender multi-turn interactions
• Developing feedback mechanisms to maximize user retention and engagement

Innovation

• Dual-view architecture for multi-turn interactions
• User simulator generates reflective engagement instructions
• Multi-agent simulations with diverse content representations