ShopSimulator: Evaluating and Exploring RL-Driven LLM Agent for Shopping Assistants

2026-01-26

AI Summary

This work addresses the absence of a unified Chinese e-commerce simulation environment and comprehensive evaluation platform that simultaneously supports personalized preference modeling, multi-turn dialogue, and high-similarity product retrieval. We propose ShopSimulator—the first large-scale Chinese shopping simulation framework that integrates training and evaluation—to systematically assess the search, conversational, and decision-making capabilities of large language model (LLM)-driven shopping assistants across diverse scenarios. By constructing user preference models and long-horizon interactive tasks, we uncover critical bottlenecks in LLMs’ ability to perform deep search and leverage personalized cues. Furthermore, we demonstrate the effectiveness of a joint training strategy combining supervised fine-tuning (SFT) and reinforcement learning (RL), which significantly improves end-to-end task success rates from below 40%, offering both methodological insights and practical guidance for developing efficient shopping assistants.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Learning Human Values and Preferences

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Large language model (LLM)-based agents are increasingly deployed in e-commerce shopping. To perform thorough, user-tailored product searches, agents should interpret personal preferences, engage in multi-turn dialogues, and ultimately retrieve and discriminate among highly similar products. However, existing research has yet to provide a unified simulation environment that consistently captures all of these aspects, and always focuses solely on evaluation benchmarks without training support. In this paper, we introduce ShopSimulator, a large-scale and challenging Chinese shopping environment. Leveraging ShopSimulator, we evaluate LLMs across diverse scenarios, finding that even the best-performing models achieve less than 40% full-success rate. Error analysis reveals that agents struggle with deep search and product selection in long trajectories, fail to balance the use of personalization cues, and to effectively engage with users. Further training exploration provides practical guidance for overcoming these weaknesses, with the combination of supervised fine-tuning (SFT) and reinforcement learning (RL) yielding significant performance improvements. Code and data will be released at https://github.com/ShopAgent-Team/ShopSimulator.

Problem

• LLM-based agents
• shopping assistant
• simulation environment
• personalized preference
• product selection

Innovation

• ShopSimulator
• LLM agent
• reinforcement learning
• shopping assistant
• simulation environment