Deadlock-Free Hybrid RL-MAPF Framework for Zero-Shot Multi-Robot Navigation

2025-11-27

AI Summary

Multi-robot navigation in cluttered, narrow environments suffers from frequent deadlocks, difficulty balancing real-time collision avoidance with long-term goal achievement, and poor generalization of reinforcement learning policies. Method: This paper proposes a hybrid navigation framework integrating deep reinforcement learning (DRL) and multi-agent path finding (MAPF). It introduces a lightweight, progress-monitoring–based deadlock detection mechanism that dynamically triggers local MAPF coordination—enabling zero-shot topological deadlock resolution without retraining. DRL handles reactive, real-time obstacle avoidance; MAPF computes globally conflict-free trajectories; and a safety layer adaptively regulates waypoint progression. Results: Experiments in high-density, heterogeneous multi-robot scenarios achieve near 100% task completion, with substantial reductions in deadlock and collision rates. The framework demonstrates strong robustness and zero-shot generalization across diverse robot configurations and environmental layouts.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Multiagent Planning
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Multi-robot navigation in cluttered environments presents fundamental challenges in balancing reactive collision avoidance with long-range goal achievement. When navigating through narrow passages or confined spaces, deadlocks frequently emerge that prevent agents from reaching their destinations, particularly when Reinforcement Learning (RL) control policies encounter novel configurations out of learning distribution. Existing RL-based approaches suffer from limited generalization capability in unseen environments. We propose a hybrid framework that seamlessly integrates RL-based reactive navigation with on-demand Multi-Agent Path Finding (MAPF) to explicitly resolve topological deadlocks. Our approach integrates a safety layer that monitors agent progress to detect deadlocks and, when detected, triggers a coordination controller for affected agents. The framework constructs globally feasible trajectories via MAPF and regulates waypoint progression to reduce inter-agent conflicts during navigation. Extensive evaluation on dense multi-agent benchmarks shows that our method boosts task completion from marginal to near-universal success, markedly reducing deadlocks and collisions. When integrated with hierarchical task planning, it enables coordinated navigation for heterogeneous robots, demonstrating that coupling reactive RL navigation with selective MAPF intervention yields a robust, zero-shot performance.

Problem

• Addresses deadlock issues in multi-robot navigation in cluttered environments
• Improves generalization of RL policies in unseen or novel configurations
• Integrates reactive RL with MAPF to resolve topological deadlocks explicitly

Innovation

• Hybrid RL-MAPF framework resolves deadlocks
• Safety layer detects deadlocks and triggers coordination
• MAPF constructs global trajectories to reduce conflicts