Learning Memory-Enhanced Improvement Heuristics for Flexible Job Shop Scheduling

2026-03-03

AI Summary

This work addresses the limitation of existing deep reinforcement learning (DRL) approaches in approximating optimal solutions for the flexible job shop scheduling problem (FJSP). To overcome this challenge, the authors propose the MIStar framework, which first formulates scheduling states using a heterogeneous disjunctive graph for precise modeling. A memory-augmented heterogeneous graph neural network (MHGNN) is then designed to enhance feature extraction and decision-making capabilities. Coupled with a parallel greedy search strategy, the framework enables efficient iterative optimization. Notably, MIStar innovatively integrates a memory mechanism into an improved DRL paradigm, significantly boosting both search efficiency and solution quality. Experimental results demonstrate that MIStar consistently outperforms conventional heuristic algorithms and state-of-the-art DRL-based constructive methods on both synthetic datasets and established public benchmarks.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

The rise of smart manufacturing under Industry 4.0 introduces mass customization and dynamic production, demanding more advanced and flexible scheduling techniques. The flexible job-shop scheduling problem (FJSP) has attracted significant attention due to its complex constraints and strong alignment with real-world production scenarios. Current deep reinforcement learning (DRL)-based approaches to FJSP predominantly employ constructive methods. While effective, they often fall short of reaching (near-)optimal solutions. In contrast, improvement-based methods iteratively explore the neighborhood of initial solutions and are more effective in approaching optimality. However, the flexible machine allocation in FJSP poses significant challenges to the application of this framework, including accurate state representation, effective policy learning, and efficient search strategies. To address these challenges, this paper proposes a Memory-enhanced Improvement Search framework with heterogeneous graph representation--MIStar. It employs a novel heterogeneous disjunctive graph that explicitly models the operation sequences on machines to accurately represent scheduling solutions. Moreover, a memoryenhanced heterogeneous graph neural network (MHGNN) is designed for feature extraction, leveraging historical trajectories to enhance the decision-making capability of the policy network. Finally, a parallel greedy search strategy is adopted to explore the solution space, enabling superior solutions with fewer iterations. Extensive experiments on synthetic data and public benchmarks demonstrate that MIStar significantly outperforms both traditional handcrafted improvement heuristics and state-of-the-art DRL-based constructive methods.

Problem

• Flexible Job Shop Scheduling
• Improvement Heuristics
• Deep Reinforcement Learning
• State Representation
• Search Strategy

Innovation

• Memory-enhanced reinforcement learning
• Heterogeneous disjunctive graph
• Improvement heuristic
• Flexible job shop scheduling
• Graph neural network