A Universal Approach to Feature Representation in Dynamic Task Assignment Problems

2025-07-04

AI Summary

Traditional reinforcement learning (RL) struggles with dynamic task allocation due to infinite state and action spaces arising from high-dimensional or continuous task and resource features. Method: This paper proposes a general feature representation framework based on *assignment graphs*, which uniformly models tasks, resources, and their heterogeneous features as structured graph data. We establish a formal mapping from colored Petri nets to assignment graphs and design a graph-adapted variant of the Proximal Policy Optimization (PPO) algorithm for end-to-end policy learning. Contribution/Results: To our knowledge, this is the first RL-based approach enabling learnable task allocation under provably infinite state and action spaces. Empirical evaluation across three canonical dynamic allocation problems demonstrates that the learned policies closely approximate optimal performance while exhibiting strong generalization across unseen task/resource configurations and scalability to large-scale instances.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Dynamic task assignment concerns the optimal assignment of resources to tasks in a business process. Recently, Deep Reinforcement Learning (DRL) has been proposed as the state of the art for solving assignment problems. DRL methods usually employ a neural network (NN) as an approximator for the policy function, which ingests the state of the process and outputs a valuation of the possible assignments. However, representing the state and the possible assignments so that they can serve as inputs and outputs for a policy NN remains an open challenge, especially when tasks or resources have features with an infinite number of possible values. To solve this problem, this paper proposes a method for representing and solving assignment problems with infinite state and action spaces. In doing so, it provides three contributions: (I) A graph-based feature representation of assignment problems, which we call assignment graph; (II) A mapping from marked Colored Petri Nets to assignment graphs; (III) An adaptation of the Proximal Policy Optimization algorithm that can learn to solve assignment problems represented through assignment graphs. To evaluate the proposed representation method, we model three archetypal assignment problems ranging from finite to infinite state and action space dimensionalities. The experiments show that the method is suitable for representing and learning close-to-optimal task assignment policies regardless of the state and action space dimensionalities.

Problem

• Representing infinite state and action spaces in task assignment
• Mapping Colored Petri Nets to graph-based feature representations
• Adapting Proximal Policy Optimization for dynamic task assignment

Innovation

• Graph-based feature representation for assignments
• Mapping Colored Petri Nets to assignment graphs
• Adapted Proximal Policy Optimization algorithm