Topology-Aware and Highly Generalizable Deep Reinforcement Learning for Efficient Retrieval in Multi-Deep Storage Systems

2025-05-27

Journal of Intelligent Manufacturing

AI Summary

In multi-depth autonomous vehicle storage/retrieval systems (AVS/RS) with heterogeneous item configurations, lane blockages cause high retrieval latency and poor scheduling flexibility. Method: This paper proposes a graph-structure-driven hybrid neural scheduling framework that innovatively integrates Graph Neural Networks (GNNs) and Transformers to jointly model local topological constraints and global priority decisions, respectively, and employs Proximal Policy Optimization (PPO)-based multi-agent reinforcement learning for deadline-aware dynamic scheduling. The framework enables cross-warehouse layout generalization without retraining. Results: Experiments across diverse real-world topologies demonstrate that the proposed method reduces average retrieval latency by 19.3%–34.7% compared to state-of-the-art heuristic approaches, significantly enhancing system real-time performance and adaptability.

Technology Category

• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In modern industrial and logistics environments, the rapid expansion of fast delivery services has heightened the demand for storage systems that combine high efficiency with increased density. Multi-deep autonomous vehicle storage and retrieval systems present a viable solution for achieving greater storage density. However, these systems encounter significant challenges during retrieval operations due to lane blockages. A conventional approach to mitigate this issue involves storing items with homogeneous characteristics in a single lane, but this strategy restricts the flexibility and adaptability of multi-deep storage systems. Building on this background, this work presents a deep reinforcement learning-based framework to optimize the retrieval process in multi-deep storage systems with heterogeneous item configurations. Each item is associated with a specific due date, and objective function is total tardiness minimization. To effectively capture the system’s topology, we introduce a graph-based state representation that integrates both item attributes and the local topological structure of the multi-deep warehouse. For processing this representation, we design a novel neural network architecture that combines a Graph Neural Network (GNN) with a Transformer model. The GNN encodes topological and item-specific information into embeddings for all directly accessible items, while the Transformer maps these embeddings into global priority assignments. The Transformer’s strong generalization capability further allows our approach to be applied to storage systems with diverse layouts. Extensive numerical experiments, including comparisons with heuristic methods, demonstrate the superiority of the proposed neural network architecture and the effectiveness of the trained agent in optimizing retrieval tardiness.

Problem

• Optimizing retrieval in multi-deep storage systems with heterogeneous items
• Minimizing total tardiness by addressing lane blockage challenges
• Developing topology-aware reinforcement learning for diverse warehouse layouts

Innovation

• Graph Neural Network for topology encoding
• Transformer model for global prioritization
• Combined GNN-Transformer architecture for generalization