🤖 AI Summary
This paper addresses the optimal execution problem under time- and size-constrained trading scenarios. Methodologically, it proposes a reinforcement learning (RL)-based autonomous decision-making framework that pioneers an end-to-end training paradigm integrating the ABIDES multi-agent market simulator with a customized Markov Decision Process (MDP) formulation. The framework dynamically learns execution policies directly from real-time limit-order-book (LOB) states, eliminating reliance on historical market data. It incorporates domain-informed LOB feature engineering and adapts both Proximal Policy Optimization (PPO) and Deep Q-Network (DQN) algorithms. Empirical evaluation demonstrates significant and robust improvements over benchmark strategies—including TWAP and VWAP—across key metrics: transaction cost (market impact), order completion rate, and timing risk. These results validate the efficacy and practicality of simulation-driven RL for algorithmic trade execution.
📝 Abstract
This study investigates the development of an optimal execution strategy through reinforcement learning, aiming to determine the most effective approach for traders to buy and sell inventory within a limited time frame. Our proposed model leverages input features derived from the current state of the limit order book. To simulate this environment and overcome the limitations associated with relying on historical data, we utilize the multi-agent market simulator ABIDES, which provides a diverse range of depth levels within the limit order book. We present a custom MDP formulation followed by the results of our methodology and benchmark the performance against standard execution strategies. Our findings suggest that the reinforcement learning-based approach demonstrates significant potential.