Compositional Coordination for Multi-Robot Teams with Large Language Models

2025-07-21

AI Summary

Traditional multi-robot collaboration relies on domain experts to manually translate natural-language tasks into mathematical models and executable code—resulting in labor-intensive workflows, poor generalizability, and limited accessibility for non-experts. Method: We propose LAN2CB, the first end-to-end framework that automatically generates task-dependent, collaborative control code for multi-robot systems directly from natural-language instructions. It comprises two stages: (i) task decomposition via hierarchical task graph modeling and structured knowledge base retrieval, and (ii) large language model–driven code generation. Contribution/Results: We introduce the first natural-language task dataset specifically designed for multi-robot collaboration. Extensive evaluation in simulation and on real robotic platforms demonstrates LAN2CB’s capability to parse diverse tasks and deploy executable code, significantly reducing manual coding effort while enhancing system adaptability and accessibility.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Intelligent Robots: Multi-Robot Systems
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-robot coordination has traditionally relied on a task-specific and expert-driven pipeline, where natural language mission descriptions are manually translated by domain experts into mathematical formulation, algorithm design, and executable code. This conventional process is labor-intensive, inaccessible to non-experts, and inflexible to changes in mission requirements. Here, we propose LAN2CB (Language to Collective Behavior), a novel framework that leverages large language models (LLMs) to streamline and generalize the multi-robot coordination pipeline. LAN2CB directly converts natural language mission descriptions into executable Python code for multi-robot systems through two key components: (1) Mission Decomposition for Task Representation, which parses the mission into a task graph with dependencies, and (2) Code Generation, which uses the task graph and a structured knowledge base to generate deployable robot control code. We further introduce a dataset of natural language mission specifications to support development and benchmarking. Experimental results in both simulation and real-world settings show that LAN2CB enables effective and flexible multi-robot coordination from natural language, significantly reducing the need for manual engineering while supporting generalization across mission types. Website: https://sites.google.com/view/lan2cb.

Problem

• Automates multi-robot coordination from natural language
• Reduces manual engineering in robot control code
• Enhances flexibility across diverse mission types

Innovation

• LLMs convert language to robot code
• Mission decomposition into task graphs
• Structured knowledge base for code generation