Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planning

2025-02-18

AI Summary

This paper addresses the adaptive decision-making challenge in main stowage planning (MPP) for container shipping under cargo demand uncertainty and dynamic vessel stability/capacity constraints. We propose a deep reinforcement learning (DRL) framework that jointly maximizes freight revenue and minimizes operational costs. A key innovation is the feasibility projection mechanism, which overcomes the fundamental limitation of conventional RL in modeling state-dependent action feasibility—enabling real-time generation of 100%-feasible policies under multistage stochastic optimization. Crucially, dynamic stability and hold-capacity constraints are explicitly embedded into the neural network architecture, eliminating the need for post-hoc feasibility correction. Evaluated on realistic shipping scenarios, our method significantly outperforms both mixed-integer programming and feasibility-regularized RL baselines, achieving superior performance in revenue, slot utilization, and policy feasibility.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Reinforcement learning (RL) has shown promise in solving various combinatorial optimization problems. However, conventional RL faces challenges when dealing with real-world constraints, especially when action space feasibility is explicit and dependent on the corresponding state or trajectory. In this work, we focus on using RL in container shipping, often considered the cornerstone of global trade, by dealing with the critical challenge of master stowage planning. The main objective is to maximize cargo revenue and minimize operational costs while navigating demand uncertainty and various complex operational constraints, namely vessel capacity and stability, which must be dynamically updated along the vessel's voyage. To address this problem, we implement a deep reinforcement learning framework with feasibility projection to solve the master stowage planning problem (MPP) under demand uncertainty. The experimental results show that our architecture efficiently finds adaptive, feasible solutions for this multi-stage stochastic optimization problem, outperforming traditional mixed-integer programming and RL with feasibility regularization. Our AI-driven decision-support policy enables adaptive and feasible planning under uncertainty, optimizing operational efficiency and capacity utilization while contributing to sustainable and resilient global supply chains.

Problem

• Optimize container shipping stowage planning
• Address demand uncertainty in shipping
• Enhance operational efficiency and capacity utilization

Innovation

• Deep Reinforcement Learning
• Feasibility Projection
• Dynamic Demand Uncertainty