Lang2Manip: A Tool for LLM-Based Symbolic-to-Geometric Planning for Manipulation

2025-12-18

AI Summary

This work addresses the fragmentation between symbolic planning and geometric motion planning in language-to-robotic-manipulation trajectory generation. We propose a robot-agnostic, planner-agnostic, and task-agnostic language-driven Task and Motion Planning (TAMP) paradigm. Methodologically, we introduce the first plug-and-play integration of large language models (LLMs) with the Kautham motion planning framework: natural language instructions are parsed into executable action sequences via symbolic logic reasoning, then coupled with collision-aware, kinematically and dynamically constrained trajectory optimization for end-to-end trajectory synthesis. Evaluated in a ROS-compatible robotic arm simulation environment, our system enables zero-code adaptation across multi-arm setups and diverse scenarios. It significantly improves the success rate of instruction-to-executable-trajectory translation and cross-task generalization. To our knowledge, this is the first fully stackable, physically grounded robotic manipulation planning framework driven entirely by LLMs.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Simulation is essential for developing robotic manipulation systems, particularly for task and motion planning (TAMP), where symbolic reasoning interfaces with geometric, kinematic, and physics-based execution. Recent advances in Large Language Models (LLMs) enable robots to generate symbolic plans from natural language, yet executing these plans in simulation often requires robot-specific engineering or planner-dependent integration. In this work, we present a unified pipeline that connects an LLM-based symbolic planner with the Kautham motion planning framework to achieve generalizable, robot-agnostic symbolic-to-geometric manipulation. Kautham provides ROS-compatible support for a wide range of industrial manipulators and offers geometric, kinodynamic, physics-driven, and constraint-based motion planning under a single interface. Our system converts language instructions into symbolic actions and computes and executes collision-free trajectories using any of Kautham's planners without additional coding. The result is a flexible and scalable tool for language-driven TAMP that is generalized across robots, planning modalities, and manipulation tasks.

Problem

• Connects LLM-based symbolic planning with geometric motion planning
• Achieves robot-agnostic symbolic-to-geometric manipulation in simulation
• Provides a flexible, scalable tool for language-driven task and motion planning

Innovation

• LLM-based symbolic planner connects to Kautham framework
• Converts language to symbolic actions and collision-free trajectories
• Generalizable, robot-agnostic tool for manipulation planning