Trajectory Adaptation using Large Language Models

2025-04-17

AI Summary

Real-time robotic trajectory adaptation to dynamic human instructions remains challenging due to poor generalization and limited interpretability. Method: We propose a language-grounded trajectory adjustment framework that directly leverages pre-trained large language models (LLMs) to generate executable code-based policies—enabling semantic understanding and numerically grounded refinement of planner outputs or demonstration trajectories, without task-specific fine-tuning. Our approach integrates code-generation–driven policy modeling, multi-platform simulation (PyBullet/Gazebo), generic motion planners (RRT/A*), and human demonstration transfer. Results: Evaluated on simulated robotic arms, UAVs, and ground robots, the framework achieves high-precision trajectory adaptation to complex, numerically parameterized multi-step instructions (e.g., “detour 0.3 m around the right-side obstacle and decelerate to stop”). It significantly outperforms existing feature-driven sequential models, demonstrating strong generalization, inherent interpretability, and natural interactive feedback capability.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Adapting robot trajectories based on human instructions as per new situations is essential for achieving more intuitive and scalable human-robot interactions. This work proposes a flexible language-based framework to adapt generic robotic trajectories produced by off-the-shelf motion planners like RRT, A-star, etc, or learned from human demonstrations. We utilize pre-trained LLMs to adapt trajectory waypoints by generating code as a policy for dense robot manipulation, enabling more complex and flexible instructions than current methods. This approach allows us to incorporate a broader range of commands, including numerical inputs. Compared to state-of-the-art feature-based sequence-to-sequence models which require training, our method does not require task-specific training and offers greater interpretability and more effective feedback mechanisms. We validate our approach through simulation experiments on the robotic manipulator, aerial vehicle, and ground robot in the Pybullet and Gazebo simulation environments, demonstrating that LLMs can successfully adapt trajectories to complex human instructions.

Problem

• Adapting robot trajectories using human instructions
• Generating code via LLMs for flexible manipulation
• Enabling complex commands without task-specific training

Innovation

• LLM-based trajectory adaptation framework
• Code generation for robot manipulation
• No task-specific training required