SayCoNav: Utilizing Large Language Models for Adaptive Collaboration in Decentralized Multi-Robot Navigation

2025-05-19

AI Summary

Heterogeneous multi-robot collaborative navigation in unknown large-scale environments suffers from poor adaptability and inflexible coordination strategies. Method: This paper proposes the first decentralized, LLM-based adaptive collaboration framework, leveraging LLM prompt engineering for task-driven dynamic role assignment, state-aware collaborative policy generation, and online re-planning; it further integrates semantic communication with MultiON-based multi-object navigation modeling to enable semantic-level inter-agent information sharing in heterogeneous robot simulations. Contribution/Results: Experiments on Multi-Object Navigation demonstrate up to a 44.28% improvement in search efficiency, alongside significantly enhanced generalization to dynamic environmental changes and team reconfiguration, as well as superior real-time adaptability.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Adaptive collaboration is critical to a team of autonomous robots to perform complicated navigation tasks in large-scale unknown environments. An effective collaboration strategy should be determined and adapted according to each robot's skills and current status to successfully achieve the shared goal. We present SayCoNav, a new approach that leverages large language models (LLMs) for automatically generating this collaboration strategy among a team of robots. Building on the collaboration strategy, each robot uses the LLM to generate its plans and actions in a decentralized way. By sharing information to each other during navigation, each robot also continuously updates its step-by-step plans accordingly. We evaluate SayCoNav on Multi-Object Navigation (MultiON) tasks, that require the team of the robots to utilize their complementary strengths to efficiently search multiple different objects in unknown environments. By validating SayCoNav with varied team compositions and conditions against baseline methods, our experimental results show that SayCoNav can improve search efficiency by at most 44.28% through effective collaboration among heterogeneous robots. It can also dynamically adapt to the changing conditions during task execution.

Problem

• Enabling adaptive collaboration in decentralized multi-robot navigation
• Automating strategy generation using large language models for heterogeneous robots
• Improving search efficiency in unknown environments through dynamic adaptation

Innovation

• Leverages LLMs for decentralized robot collaboration
• Dynamically adapts plans based on shared information
• Improves search efficiency via heterogeneous robot teamwork