DEXTER-LLM: Dynamic and Explainable Coordination of Multi-Robot Systems in Unknown Environments via Large Language Models

2025-08-19

AI Summary

To address the insufficient adaptability and interpretability of online multi-robot collaborative task planning in open, unknown environments, this paper proposes a four-module synergistic framework: (1) LLM-enhanced semantic understanding and multi-stage task decomposition; (2) Linear Temporal Logic (LTL)-driven formal specification modeling; (3) search-based optimal subtask allocation; and (4) multi-rate event-driven dynamic re-planning. The framework supports dual-modal input—natural language and LTL—and enables real-time task generation, human-in-the-loop verification, and online updates triggered by semantic features. Experiments demonstrate 100% task success rate, an average completion of 160 composite tasks (triple the baseline), a 62% reduction in LLM queries, and a twofold improvement in planning quality. The approach significantly enhances adaptability and decision interpretability in open environments.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Multiagent Planning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Online coordination of multi-robot systems in open and unknown environments faces significant challenges, particularly when semantic features detected during operation dynamically trigger new tasks. Recent large language model (LLMs)-based approaches for scene reasoning and planning primarily focus on one-shot, end-to-end solutions in known environments, lacking both dynamic adaptation capabilities for online operation and explainability in the processes of planning. To address these issues, a novel framework (DEXTER-LLM) for dynamic task planning in unknown environments, integrates four modules: (i) a mission comprehension module that resolves partial ordering of tasks specified by natural languages or linear temporal logic formulas (LTL); (ii) an online subtask generator based on LLMs that improves the accuracy and explainability of task decomposition via multi-stage reasoning; (iii) an optimal subtask assigner and scheduler that allocates subtasks to robots via search-based optimization; and (iv) a dynamic adaptation and human-in-the-loop verification module that implements multi-rate, event-based updates for both subtasks and their assignments, to cope with new features and tasks detected online. The framework effectively combines LLMs' open-world reasoning capabilities with the optimality of model-based assignment methods, simultaneously addressing the critical issue of online adaptability and explainability. Experimental evaluations demonstrate exceptional performances, with 100% success rates across all scenarios, 160 tasks and 480 subtasks completed on average (3 times the baselines), 62% less queries to LLMs during adaptation, and superior plan quality (2 times higher) for compound tasks. Project page at https://tcxm.github.io/DEXTER-LLM/

Problem

• Dynamic task planning for multi-robot systems in unknown environments
• Improving explainability and adaptation in online semantic task decomposition
• Integrating LLM reasoning with optimal model-based assignment methods

Innovation

• Dynamic task planning via four integrated modules
• LLM-based subtask generator for explainable reasoning
• Multi-rate event updates for online adaptation