Multi-Level Market Making with Reinforcement Learning

2026-08-18

AI Summary

研究使用强化学习框架解决多级市场做市问题，通过动态提交不同规模的市价和限价订单并控制库存大小以最大化交易收入。

Abstract

We introduce a reinforcement learning framework for market making in a limit order book. Our algorithm aims to maximize trading revenue by dynamically submitting market and limit orders of varying sizes across multiple price levels while controlling inventory size. We use multivariate logistic-normal distributions to model order allocations and employ a deep-set encoder to aggregate features from variable-length order sets into a fixed-dimensional latent representation. Additionally, we incorporate potential-based reward shaping to accelerate learning without altering the optimal policy. We illustrate the performance of the method in three simulated market environments consisting of noise traders who submit random trades, tactical traders who respond to instantaneous volume imbalance, and strategic traders who trade in the direction of an exponentially weighted volume imbalance signal.

Problem

• Market Making
• Reinforcement Learning
• Limit Order Book
• Trading Revenue
• Inventory Control

Innovation

• Reinforcement Learning
• Multi-Level Market Making
• Deep-Set Encoder
• Potential-Based Reward Shaping