Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets

2025-10-27

AI Summary

Systematic empirical study of large language model (LLM) agents’ economic behavior in realistic markets remains lacking; existing work is largely confined to single-task or structured two-agent settings, failing to capture authentic interactions within open, dynamic, multi-agent ecosystems. Method: We introduce the first open-source bilateral agent market simulation framework, enabling large-scale, heterogeneous market emulation where consumer-assistant and service-provider agents engage in multi-turn open-ended dialogues. Contribution/Results: Our framework features a scalable simulation architecture, adversarial interaction design, and a multi-dimensional evaluation protocol. Experiments reveal a pronounced “first-proposal bias”—where response speed dominates quality—and show that state-of-the-art LLMs approach optimal social welfare only under idealized search conditions. Performance degrades sharply with scale, and market outcomes prove highly sensitive to search mechanisms and first-mover advantages.

Technology Category

• Multiagent Systems: Mechanism Design
• Search and Optimization: Sampling/Simulation-based Search
• Game Theory and Economic Paradigms: Auctions and Market-Based Systems

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

As LLM agents advance, they are increasingly mediating economic decisions, ranging from product discovery to transactions, on behalf of users. Such applications promise benefits but also raise many questions about agent accountability and value for users. Addressing these questions requires understanding how agents behave in realistic market conditions. However, previous research has largely evaluated agents in constrained settings, such as single-task marketplaces (e.g., negotiation) or structured two-agent interactions. Real-world markets are fundamentally different: they require agents to handle diverse economic activities and coordinate within large, dynamic ecosystems where multiple agents with opaque behaviors may engage in open-ended dialogues. To bridge this gap, we investigate two-sided agentic marketplaces where Assistant agents represent consumers and Service agents represent competing businesses. To study these interactions safely, we develop Magentic-Marketplace-- a simulated environment where Assistants and Services can operate. This environment enables us to study key market dynamics: the utility agents achieve, behavioral biases, vulnerability to manipulation, and how search mechanisms shape market outcomes. Our experiments show that frontier models can approach optimal welfare-- but only under ideal search conditions. Performance degrades sharply with scale, and all models exhibit severe first-proposal bias, creating 10-30x advantages for response speed over quality. These findings reveal how behaviors emerge across market conditions, informing the design of fair and efficient agentic marketplaces.

Problem

• Studying agent behavior in realistic two-sided marketplaces with dynamic interactions
• Investigating how search mechanisms and scale affect agent performance and welfare
• Analyzing behavioral biases like first-proposal bias in agent-mediated economic decisions

Innovation

• Simulated environment for agentic marketplaces study
• Two-sided marketplace with consumer and business agents
• Analyzes market dynamics and behavioral biases safely