ProcureGym: A Multi-Agent Markov Game Framework for Modeling National Volume-based Drug Procurement

2026-03-24

AI Summary

This study addresses the strategic bidding behavior of multiple firms and policy impact evaluation in China’s national centralized pharmaceutical volume-based procurement program. It formulates the problem as a multi-agent Markov game for the first time and develops a high-fidelity simulation platform grounded in seven rounds of real-world procurement data, encompassing 325 drugs and 2,267 enterprises. The proposed approach integrates multi-agent reinforcement learning, large language models, and rule-based strategies, significantly outperforming baseline methods in both bid-winning consistency and profitability. The analysis identifies the highest effective bid price and procurement volume as key determinants of bidding outcomes, offering data-driven insights to inform policy optimization.

Technology Category

• Multiagent Systems: Mechanism Design
• Game Theory and Economic Paradigms: Auctions and Market-Based Systems
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In this paper, we introduce ProcureGym, an data-driven multi-agent simulation platform that models China's National Volume-Based drug Procurement (NVBP) as a Markov Game. Based on real-world data from 7 rounds of NVBP (covering 325 drugs and 2,267 firms), the platform establishes a high-fidelity simulation environment. Within this framework, we evaluate diverse agent models, including Reinforcement Learning (RL), Large Language Model (LLM), and Rule-based algorithms. Experimental results demonstrate that RL agents achieve superior winner alignment and profits. Further analyses show that maximum valid bidding price and procurement volume dominate strategic outcomes. ProcureGym thus serves as a rigorous instrument for assessing policy impacts and formulating future procurement strategies.

Problem

• National Volume-Based Drug Procurement
• Multi-Agent Simulation
• Markov Game
• Bidding Strategy
• Healthcare Policy

Innovation

• Multi-Agent Reinforcement Learning
• Markov Game
• Volume-Based Procurement
• Large Language Model
• Healthcare Policy Simulation