GymPN: A Library for Decision-Making in Process Management Systems

2025-06-25

AI Summary

This paper addresses the joint decision-making problem in business process management—encompassing task sequencing, execution timing, and resource allocation—by proposing a deep reinforcement learning (DRL)-based modeling and solution framework. Methodologically, it formalizes complex processes as partially observable Markov decision processes (POMDPs), enabling heterogeneous decision coupling and incorporation of real-world constraints; it further provides an extensible open-source library for end-to-end policy learning. The key contribution lies in being the first to unify multi-granularity, multi-objective decisions within process management, while overcoming traditional methods’ reliance on full observability and static process structures. Experiments across eight representative business process patterns demonstrate that the model accurately captures dynamic environmental characteristics, significantly improving process performance—e.g., reducing cycle time by 18.7% and increasing resource utilization by 23.4%—while exhibiting strong generalization capability and deployment efficiency.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Reinforcement Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Process management systems support key decisions about the way work is allocated in organizations. This includes decisions on which task to perform next, when to execute the task, and who to assign the task to. Suitable software tools are required to support these decisions in a way that is optimal for the organization. This paper presents a software library, called GymPN, that supports optimal decision-making in business processes using Deep Reinforcement Learning. GymPN builds on previous work that supports task assignment in business processes, introducing two key novelties: support for partial process observability and the ability to model multiple decisions in a business process. These novel elements address fundamental limitations of previous work and thus enable the representation of more realistic process decisions. We evaluate the library on eight typical business process decision-making problem patterns, showing that GymPN allows for easy modeling of the desired problems, as well as learning optimal decision policies.

Problem

• Supports optimal decision-making in business processes
• Addresses partial process observability in workflows
• Models multiple decisions within a single process

Innovation

• Deep Reinforcement Learning for decision-making
• Supports partial process observability
• Models multiple decisions in processes