DM$^3$-Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

2026-04-23

AI Summary

This work addresses the challenge of open-vocabulary, multi-target semantic navigation for multi-robot systems operating without centralized coordination or shared global maps. The authors propose a fully decentralized multi-agent framework that enables efficient autonomous collaboration through peer-to-peer asynchronous communication of local maps and navigation intents. By integrating an implicit task allocation mechanism, a distance-weighted frontier selection strategy, and open-vocabulary semantic understanding, the system eliminates the need for centralized scheduling, thereby reducing redundant exploration and avoiding single points of failure. Experimental results demonstrate that the approach matches or surpasses centralized baselines on the HM3DSem and GOAT-Bench benchmarks, while real-world deployment with two onboard robots in an office environment confirms its practical feasibility.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Coordination and Collaboration
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present DM$^3$-Nav, a fully decentralized multi-agent semantic navigation system supporting multimodal open-vocabulary goal specification and multi-object missions. In our setting, decentralization implies operation without a central coordinator, global map aggregation, or shared global state at runtime. Robots operate autonomously and coordinate through ad-hoc pairwise communication, exchanging local maps, goal status, and navigation intent without synchronization. An implicit task allocation mechanism combining intent broadcasting and distance-weighted frontier selection reduces redundant exploration while preserving decentralized operation. Evaluations on HM3DSem scenes using the HM3Dv0.2 and GOAT-Bench datasets demonstrate that DM$^3$-Nav matches or exceeds centralized and shared-map baselines while eliminating single points of failure inherent in centralized architectures. Finally, we validate our approach in a real-world office environment using two mobile robots, demonstrating successful deployment relying entirely on onboard sensing and computation. A video of our real-world experiments is available online: https://drive.google.com/file/d/1QiUSCn5rIvtuTUqtuXLPgmt6S8x9-MCZ/view?usp=drive_link

Problem

• decentralized
• multi-agent
• semantic navigation
• multimodal
• multi-object

Innovation

• decentralized multi-agent navigation
• semantic navigation
• open-vocabulary goal specification
• implicit task allocation
• onboard autonomy