DAG-Plan: Generating Directed Acyclic Dependency Graphs for Dual-Arm Cooperative Planning

2024-06-14

arXiv.org

AI Summary

Long-horizon collaborative tasks for dual robotic arms face challenges including complex spatiotemporal dependencies among subtasks, difficulty in dynamic action allocation, and limited expressiveness of linear programming formulations. This paper proposes the first LLM-driven DAG-structured task decomposition framework, which automatically parses high-level instructions into directed acyclic graphs (DAGs) encoding dependency constraints, and integrates environment perception to enable real-time, dynamic action allocation and parallel adaptive execution across both arms. The method breaks away from predefined operational paradigms, supporting end-to-end, interpretable, and generalizable collaborative planning. Evaluated on the Dual-Arm Kitchen benchmark, it achieves a 52.8% efficiency gain over single-arm systems, improves success rate by 48% and reduces LLM query count by 84.1% compared to conventional dual-arm planners, significantly enhancing robustness and scalability in complex scenarios.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Dual-arm robots offer enhanced versatility and efficiency over single-arm counterparts by enabling concurrent manipulation of multiple objects or cooperative execution of tasks using both arms. However, the coordination of dual-arm systems for long-horizon tasks continues to pose significant challenges, stemming from the intricate temporal and spatial dependencies among sub-tasks, necessitating intelligent decisions regarding the allocation of actions between arms and their optimal execution order. Existing task planning methods predominantly focus on single-arm robots or rely on predefined bimanual operations to use large language models (LLMs) generate task sequence with linear temporal dependency, failing to fully leverage the capabilities of dual-arm systems. To address this limitation, we introduce DAG-Plan, a structured task planning framework tailored for dual-arm robots. DAG-Plan harnesses LLMs to decompose intricate tasks into actionable sub-tasks represented as nodes within a directed acyclic graph (DAG). Critically, DAG-Plan dynamically assigns these sub-tasks to the appropriate arm based on real-time environmental observations, enabling parallel and adaptive execution. We evaluate DAG-Plan on the Dual-Arm Kitchen Benchmark, comprising 5 sequential tasks with 44 sub-tasks. Extensive experiments demonstrate the superiority of DAG-Plan over directly using LLM to generate linear task sequence, achieving 52.8% higher efficiency compared to the single-arm task planning and 48% higher success rate of the dual-arm task planning. Compared to iterative methods, DAG-Plan improving execution efficiency 84.1% due to its fewer query time. More demos and information are available on https://sites.google.com/view/dag-plan.

Problem

• Coordinating dual-arm robots for complex long-horizon tasks
• Managing temporal and spatial dependencies between sub-tasks
• Optimizing action allocation and execution order for dual arms

Innovation

• Uses LLMs to decompose tasks into sub-tasks
• Dynamically assigns sub-tasks to arms
• Enables parallel and adaptive execution