A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery

2026-07-03

AI Summary

This work addresses the challenges of dynamic order arrivals in multi-product coordinated delivery systems, where shifting supply dependencies necessitate real-time adjustments to feasible job-machine assignments. To tackle this, the authors propose a sliding window–based deep reinforcement learning scheduling framework. The approach formulates the scheduling process as a heterogeneous graph Markov decision process, employing a sliding window to filter inactive nodes and a spatiotemporal graph encoding network to capture bottleneck shifts. Integrated with dynamic action mapping and a constrained waiting strategy, the framework enables end-to-end online scheduling. Evaluated in a real-world home appliance manufacturing setting, the method significantly reduces order tardiness compared to classical dispatching rules and existing deep reinforcement learning approaches, demonstrating strong robustness across varying resource configurations, system loads, and order arrival densities.

Abstract

Multi-product kitting delivery imposes significant challenges for real-time scheduling in hybrid manufacturing systems that integrate processing and assembly, as dynamic order arrivals simultaneously alter supply dependencies and the set of feasible job-machine assignments. This paper proposes a sliding-window-based reinforcement learning (SWRL) framework for end-to-end online scheduling in the flexible assembly flow shop scheduling problem with complex kitting constraints. The problem is formulated as a heterogeneous graph-based Markov decision process that captures the dual-layer kitting structure and the tail-product bottleneck dynamics that produce a sparse reward landscape. To address the resulting challenges, SWRL integrates a sliding-window filtering mechanism that filters inactive nodes and prioritizes kitting-critical operations, a spatiotemporal graph encoding network that tracks bottleneck shifts across consecutive decision states, and a dynamic action mapping module with a constrained waiting strategy that adapts to the changing action space under variable topologies. Experiments on real-world instances from a home appliance manufacturer demonstrate that SWRL achieves consistent tardiness reductions over classical dispatching rules and existing deep reinforcement learning methods, and exhibits robust performance across varying resource configurations, order loads, and arrival concentrations.

Problem

• dynamic assembly flow shop scheduling
• multi-product kitting delivery
• real-time scheduling
• hybrid manufacturing systems
• sparse reward landscape

Innovation

• sliding-window reinforcement learning
• assembly flow shop scheduling
• multi-product kitting
• heterogeneous graph MDP
• dynamic action space