ABIDES-MARL: A Multi-Agent Reinforcement Learning Environment for Endogenous Price Formation and Execution in a Limit Order Book

2025-11-03

AI Summary

This paper investigates equilibrium behavior and endogenous price formation in complex financial markets under multi-agent strategic interaction. We propose a synchronized multi-agent reinforcement learning (MARL) framework that integrates a high-fidelity limit-order-book simulator—extended from ABIDES-Gym—with an augmented Kyle model. To accommodate heterogeneous agents, we decouple state observation from kernel-level interruptions, enabling parallel decision-making; execution optimization is embedded directly into the strategic policy game to achieve endogenous liquidity modeling. Our contributions are threefold: (i) the first MARL framework to rigorously preserve market microstructure constraints—including price-time priority; (ii) successful replication of gradual price discovery dynamics; and (iii) causal identification of how execution strategies shape market-maker behavior and price evolution. Empirical evaluation confirms model validity and supports reproducible equilibrium analysis.

Technology Category

• Multiagent Systems: Mechanism Design
• Machine Learning: Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We present ABIDES-MARL, a framework that combines a new multi-agent reinforcement learning (MARL) methodology with a new realistic limit-order-book (LOB) simulation system to study equilibrium behavior in complex financial market games. The system extends ABIDES-Gym by decoupling state collection from kernel interruption, enabling synchronized learning and decision-making for multiple adaptive agents while maintaining compatibility with standard RL libraries. It preserves key market features such as price-time priority and discrete tick sizes. Methodologically, we use MARL to approximate equilibrium-like behavior in multi-period trading games with a finite number of heterogeneous agents-an informed trader, a liquidity trader, noise traders, and competing market makers-all with individual price impacts. This setting bridges optimal execution and market microstructure by embedding the liquidity trader's optimization problem within a strategic trading environment. We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon. We then extend the analysis to a liquidity trader's problem where market liquidity arises endogenously and show that, at equilibrium, execution strategies shape market-maker behavior and price dynamics. ABIDES-MARL provides a reproducible foundation for analyzing equilibrium and strategic adaptation in realistic markets and contributes toward building economically interpretable agentic AI systems for finance.

Problem

• Modeling equilibrium behavior in multi-agent financial market games
• Studying endogenous price formation in limit order book systems
• Analyzing strategic trading interactions between heterogeneous market participants

Innovation

• MARL methodology for multi-agent equilibrium behavior
• Decoupled state collection enabling synchronized learning
• Simulation embedding liquidity optimization in strategic trading