Air-Ground Collaboration for Language-Specified Missions in Unknown Environments
🤖 AI Summary
This work addresses the challenge of language-driven cooperative task execution by heterogeneous aerial-ground robots (UAVs/UGVs) in unknown environments. Methodologically, we propose the first end-to-end language-driven collaborative system integrating large language model (LLM)-based real-time task planning, semantic-metric online joint mapping, opportunistic cross-platform information sharing, and robust intermittent communication coordination—enabling dynamic command updates and kilometer-scale navigation. Our key contributions are: (i) the first closed-loop control pipeline mapping natural language instructions directly to coordinated multi-robot actions; and (ii) breakthrough capabilities in maintaining semantic consistency and enabling cross-modal task replanning under communication constraints. We validate the system on seven diverse natural-language tasks across urban and rural complex environments, demonstrating effectiveness in long-range navigation, dynamic replanning, and multimodal semantic reasoning.
Application Category
📝 Abstract
As autonomous robotic systems become increasingly mature, users will want to specify missions at the level of intent rather than in low-level detail. Language is an expressive and intuitive medium for such mission specification. However, realizing language-guided robotic teams requires overcoming significant technical hurdles. Interpreting and realizing language-specified missions requires advanced semantic reasoning. Successful heterogeneous robots must effectively coordinate actions and share information across varying viewpoints. Additionally, communication between robots is typically intermittent, necessitating robust strategies that leverage communication opportunities to maintain coordination and achieve mission objectives. In this work, we present a first-of-its-kind system where an unmanned aerial vehicle (UAV) and an unmanned ground vehicle (UGV) are able to collaboratively accomplish missions specified in natural language while reacting to changes in specification on the fly. We leverage a Large Language Model (LLM)-enabled planner to reason over semantic-metric maps that are built online and opportunistically shared between an aerial and a ground robot. We consider task-driven navigation in urban and rural areas. Our system must infer mission-relevant semantics and actively acquire information via semantic mapping. In both ground and air-ground teaming experiments, we demonstrate our system on seven different natural-language specifications at up to kilometer-scale navigation.
Problem
Research questions and friction points this paper is trying to address.
Enabling language-guided collaboration between UAV and UGV in unknown environments
Overcoming intermittent communication for robust air-ground coordination
Achieving semantic reasoning for dynamic mission updates via LLM planning
Innovation
Methods, ideas, or system contributions that make the work stand out.
LLM-enabled planner for semantic reasoning
Online semantic-metric maps sharing
Robust air-ground coordination strategy
🔎 Similar Papers
No similar papers found.
