Beyond Task and Motion Planning: Hierarchical Robot Planning with General-Purpose Policies

2025-04-24

AI Summary

Traditional hierarchical robotic planning simplifies task-level actions into open-loop kinematic skills, hindering seamless integration of pre-trained closed-loop motor controllers. Method: We propose Composable Interaction Primitives (CIPs), a framework enabling plug-and-play composition of heterogeneous, non-composable pre-trained skills within task-and-motion planning. Building upon CIPs, we introduce Task-and-Skill Planning (TASP), a unified architecture that jointly models symbolic task planning, geometric motion planning, and learned closed-loop control. Contribution/Results: TASP transcends reliance on motion-centric skills by elevating task semantics to the perception–action closed-loop level. Evaluated on a real mobile manipulator, it achieves end-to-end autonomous execution of multi-step complex tasks—including dynamic collaborative transport and tool manipulation—demonstrating significantly improved skill reusability and environmental adaptability.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Intelligent Robots: Motion and Path Planning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Task and motion planning is a well-established approach for solving long-horizon robot planning problems. However, traditional methods assume that each task-level robot action, or skill, can be reduced to kinematic motion planning. In this work, we address the challenge of planning with both kinematic skills and closed-loop motor controllers that go beyond kinematic considerations. We propose a novel method that integrates these controllers into motion planning using Composable Interaction Primitives (CIPs), enabling the use of diverse, non-composable pre-learned skills in hierarchical robot planning. Toward validating our Task and Skill Planning (TASP) approach, we describe ongoing robot experiments in real-world scenarios designed to demonstrate how CIPs can allow a mobile manipulator robot to effectively combine motion planning with general-purpose skills to accomplish complex tasks.

Problem

• Integrating kinematic skills and closed-loop motor controllers in planning
• Enabling use of diverse pre-learned skills in hierarchical planning
• Combining motion planning with general-purpose skills for complex tasks

Innovation

• Integrates kinematic skills and motor controllers
• Uses Composable Interaction Primitives (CIPs)
• Combines motion planning with general-purpose skills