HGT-Scheduler: Deep Reinforcement Learning for the Job Shop Scheduling Problem via Heterogeneous Graph Transformers

2026-03-06

AI Summary

This work addresses a critical limitation in existing reinforcement learning approaches to job shop scheduling, which typically encode both precedence constraints and machine conflict relations as homogeneous graphs, thereby neglecting the problem’s inherent heterogeneous structure and leading to significant information loss. To overcome this, the paper proposes the first scheduling policy that explicitly distinguishes and models these two edge types using a heterogeneous graph Transformer. The method employs edge-type-specific attention mechanisms to separately process precedence and conflict relationships and is trained with Proximal Policy Optimization (PPO). On the FT06 benchmark, it achieves an optimality gap of 8.4%, significantly outperforming homogeneous graph models (p = 0.011) and GIN baselines. It also demonstrates strong scalability on FT10, with ablation studies confirming that a three-layer attention architecture yields optimal performance.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The Job Shop Scheduling Problem (JSSP) is commonly formulated as a disjunctive graph in which nodes represent operations and edges encode technological precedence constraints as well as machine-sharing conflicts. Most existing reinforcement learning approaches model this graph as homogeneous, merging job-precedence and machine-contention edges into a single relation type. Such a simplification overlooks the intrinsic heterogeneity of the problem structure and may lead to the loss of critical relational information. To address this limitation, we propose the Heterogeneous Graph Transformer (HGT)-Scheduler, a reinforcement learning framework that models the JSSP as a heterogeneous graph. The proposed architecture leverages a Heterogeneous Graph Transformer to capture type-specific relational patterns through edge-type-dependent attention mechanisms applied to precedence and contention relations. The scheduling policy is trained using Proximal Policy Optimization. The effectiveness of the proposed method is evaluated on the Fisher--Thompson benchmark instances. On the FT06 instance, the HGT-Scheduler achieves an optimality gap of 8.4\%, statistically outperforming both an identical architecture that ignores edge types ($p = 0.011$) and a standard Graph Isomorphism Network baseline. On the larger FT10 instance, the approach demonstrates favorable scalability. However, under a 50,000-step training limit, the performance of heterogeneous and homogeneous graph models is comparable, suggesting that edge-type awareness requires longer training horizons for larger problem instances. Ablation analyses further indicate that a three-layer attention architecture provides the best performance. Overall, the results confirm that explicitly modeling distinct edge semantics improves the learning of effective scheduling policies.

Problem

• Job Shop Scheduling Problem
• Heterogeneous Graph
• Reinforcement Learning
• Disjunctive Graph
• Edge Semantics

Innovation

• Heterogeneous Graph Transformer
• Job Shop Scheduling Problem
• Edge-type-dependent Attention
• Deep Reinforcement Learning
• Disjunctive Graph