Offline Reinforcement Learning for Warehouse SLAM Throughput Control

2026-06-22

AI Summary

This work addresses the challenge of throughput control in warehouse SLAM (Scan/Label/Apply/Ship) systems, which requires balancing increased throughput against downstream stability. The authors propose an algorithm-agnostic offline reinforcement learning framework that learns dynamic throttling policies from historical operational logs. Key innovations include a history-aware state representation, an action abstraction that accounts for delayed effects, and a reward function jointly optimizing upstream and downstream performance metrics. The framework supports integration with various offline RL algorithms. Evaluated with Conservative Q-Learning (CQL), the resulting policy significantly outperforms baseline methods, achieving a 22.97% improvement in system health and a 3.18% reduction in average throttling duration in real-world deployment, thereby demonstrating the approach’s effectiveness and scalability for throughput control in complex logistics environments.

Abstract

We present an offline reinforcement learning (RL) framework for optimizing SLAM throughput control in a warehouse fulfillment environment. SLAM (Scan/Label/Apply/Manifest) throughput directly influences system congestion and operational efficiency. Our RL-based control approach dynamically recommends SLAM throughput settings that adaptively balance throughput maximization with downstream stability through intelligent adjustment of throttling behavior. We include a history-informed state representation, action space abstraction for delayed-impact control, and a reward function that captures both upstream and downstream operational metrics. Our approach is algorithm-agnostic, enabling integration of multiple offline RL methods under a unified architecture. We instantiate our framework with three state-of-the-art offline RL algorithms, and trained the models offline using de-identified historical operational logs from a large-scale warehouse. Policy performance is evaluated using a comprehensive multi-method strategy. These include model-free approaches including immediate reward estimation via regression models and long-horizon Fitted Q Evaluation (FQE), as well as model-based Deep Koopman dynamics evaluation. Empirical results reveal that the CQL policy consistently outperforms alternatives, improving system health by 22.97% and reducing average throttling duration by 3.18%. These findings demonstrate the potential of offline RL for safe and scalable warehouse throughput control optimization.

Problem

• Offline Reinforcement Learning
• SLAM Throughput Control
• Warehouse Fulfillment
• System Congestion
• Operational Efficiency

Innovation

• offline reinforcement learning
• throughput control
• SLAM optimization
• delayed-impact control
• algorithm-agnostic framework