Collaborative motion planning for multi-manipulator systems through Reinforcement Learning and Dynamic Movement Primitives

2024-10-01

arXiv.org

AI Summary

To address challenges in multi-arm robotic collaboration—including collision avoidance, latency in real-time trajectory generation, and poor adaptability to dynamic environments—this paper proposes an RL-DMP hierarchical planning framework. At the high level, a Proximal Policy Optimization (PPO) reinforcement learning policy network enables task-level decision-making and zero-shot transfer across tasks; at the low level, Dynamic Movement Primitives (DMPs) ensure motion smoothness and millisecond-scale trajectory execution. The framework introduces, for the first time, a tightly coupled coordination mechanism between the policy and execution layers. Evaluated in PyBullet simulations with multiple UR5e manipulators performing complex collaborative tasks, it achieves an average trajectory generation latency of <50 ms. It significantly outperforms pure RL or pure DMP baselines in collision-free success rate, while demonstrating strong task generalization and real-time capability.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Robotic tasks often require multiple manipulators to enhance task efficiency and speed, but this increases complexity in terms of collaboration, collision avoidance, and the expanded state-action space. To address these challenges, we propose a multi-level approach combining Reinforcement Learning (RL) and Dynamic Movement Primitives (DMP) to generate adaptive, real-time trajectories for new tasks in dynamic environments using a demonstration library. This method ensures collision-free trajectory generation and efficient collaborative motion planning. We validate the approach through experiments in the PyBullet simulation environment with UR5e robotic manipulators.

Problem

• Enhance multi-manipulator task efficiency and speed
• Address collaboration and collision avoidance challenges
• Generate adaptive real-time trajectories in dynamic environments

Innovation

• Combines Reinforcement Learning with Dynamic Movement Primitives
• Generates real-time, adaptive trajectories for dynamic environments
• Ensures collision-free, efficient collaborative motion planning