Learning-guided Prioritized Planning for Lifelong Multi-Agent Path Finding in Warehouse Automation

2026-03-24

AI Summary

This work addresses the high complexity and dynamic adaptability challenges of lifelong multi-agent pathfinding (Lifelong MAPF) in warehouse automation by proposing a novel approach that integrates reinforcement learning with priority-based search. Within a rolling-horizon framework, the method combines a partially observable Markov decision process formulation with an attention-mechanism neural network to dynamically generate agent priority policies that adapt to real-time congestion states. Notably, this is the first effort to incorporate reinforcement learning into classical priority planning, thereby overcoming the limitations of conventional heuristics. Extensive evaluations in realistic warehouse simulations demonstrate that the system consistently achieves optimal throughput across diverse settings of agent density, layout configurations, and time horizons, highlighting its strong generalization capability and practical applicability.

Technology Category

• Multiagent Systems: Multiagent Planning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Lifelong Multi-Agent Path Finding (MAPF) is critical for modern warehouse automation, which requires multiple robots to continuously navigate conflict-free paths to optimize the overall system throughput. However, the complexity of warehouse environments and the long-term dynamics of lifelong MAPF often demand costly adaptations to classical search-based solvers. While machine learning methods have been explored, their superiority over search-based methods remains inconclusive. In this paper, we introduce Reinforcement Learning (RL) guided Rolling Horizon Prioritized Planning (RL-RH-PP), the first framework integrating RL with search-based planning for lifelong MAPF. Specifically, we leverage classical Prioritized Planning (PP) as a backbone for its simplicity and flexibility in integrating with a learning-based priority assignment policy. By formulating dynamic priority assignment as a Partially Observable Markov Decision Process (POMDP), RL-RH-PP exploits the sequential decision-making nature of lifelong planning while delegating complex spatial-temporal interactions among agents to reinforcement learning. An attention-based neural network autoregressively decodes priority orders on-the-fly, enabling efficient sequential single-agent planning by the PP planner. Evaluations in realistic warehouse simulations show that RL-RH-PP achieves the highest total throughput among baselines and generalizes effectively across agent densities, planning horizons, and warehouse layouts. Our interpretive analyses reveal that RL-RH-PP proactively prioritizes congested agents and strategically redirects agents from congestion, easing traffic flow and boosting throughput. These findings highlight the potential of learning-guided approaches to augment traditional heuristics in modern warehouse automation.

Problem

• Lifelong Multi-Agent Path Finding
• warehouse automation
• throughput optimization
• conflict-free path planning
• multi-robot coordination

Innovation

• Reinforcement Learning
• Lifelong Multi-Agent Path Finding
• Prioritized Planning
• POMDP
• Attention-based Neural Network