Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration

2024-12-24

AI Summary

To address low efficiency and high communication overhead in multi-robot collaborative semantic navigation within home environments, this paper proposes MCoCoNav—a modular multimodal chain-of-thought cooperative navigation framework. MCoCoNav introduces a novel multimodal chain-of-thought scoring mechanism for collaborative decision-making, leveraging a lightweight, shareable global semantic map as the communication hub. It integrates visual perception with vision-language models (VLMs) to probabilistically assess exploration value, supports decentralized dynamic frontier selection and historical node revisiting, and jointly optimizes exploration guidance and semantic consistency. Evaluated on HM3D_v0.2 and MP3D datasets, MCoCoNav significantly improves navigation success rate and exploration coverage while reducing task completion time and communication bandwidth consumption. These results demonstrate its effectiveness, robustness, and scalability in multi-robot semantic navigation.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Understanding how humans cooperatively utilize semantic knowledge to explore unfamiliar environments and decide on navigation directions is critical for house service multi-robot systems. Previous methods primarily focused on single-robot centralized planning strategies, which severely limited exploration efficiency. Recent research has considered decentralized planning strategies for multiple robots, assigning separate planning models to each robot, but these approaches often overlook communication costs. In this work, we propose Multimodal Chain-of-Thought Co-Navigation (MCoCoNav), a modular approach that utilizes multimodal Chain-of-Thought to plan collaborative semantic navigation for multiple robots. MCoCoNav combines visual perception with Vision Language Models (VLMs) to evaluate exploration value through probabilistic scoring, thus reducing time costs and achieving stable outputs. Additionally, a global semantic map is used as a communication bridge, minimizing communication overhead while integrating observational results. Guided by scores that reflect exploration trends, robots utilize this map to assess whether to explore new frontier points or revisit history nodes. Experiments on HM3D_v0.2 and MP3D demonstrate the effectiveness of our approach. Our code is available at https://github.com/FrankZxShen/MCoCoNav.git.

Problem

• Multi-Robot Cooperation
• Environmental Exploration
• Language-Informed Path Planning

Innovation

• MCoCoNav
• Visual-Linguistic Collaborative Navigation
• Multi-Robot Exploration