Reinforcement Learning of Multi-robot Task Allocation for Multi-object Transportation with Infeasible Tasks

2024-04-18

IEEE/SICE International Symposium on System Integration

AI Summary

In multi-robot cooperative transport, deadlocks arise when object weights are unknown and infeasible tasks (e.g., exceeding payload capacity) are assigned without prior feasibility assessment. Method: This paper proposes a dynamic, scalable task allocation framework featuring a novel cloud-based broadcast mechanism for shared task experience and an individualized exclusion-rank learning model. Integrating reinforcement learning with distributed experience modeling, the framework enables real-time task re-prioritization and transient task exclusion—without requiring prior knowledge of task feasibility—and supports hybrid collaborative/independent transport strategies as well as on-the-fly robot addition. Results: Experiments demonstrate that the framework maintains high generalizability and scalability under increasing numbers of robots and objects, effectively prevents transient deadlocks, and exhibits robust, efficient allocation performance even for objects with previously unseen weights.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Multi-object transport using multi-robot systems has the potential for diverse practical applications such as delivery services owing to its efficient individual and scalable cooperative transport. However, allocating transportation tasks of objects with unknown weights remains challenging. Moreover, the presence of infeasible tasks (untransportable objects) can lead to robot stoppage (deadlock). This paper proposes a framework for dynamic task allocation that involves storing task experiences for each task in a scalable manner with respect to the number of robots. First, these experiences are broadcasted from the cloud server to the entire robot system. Subsequently, each robot learns the exclusion levels for each task based on those task experiences, enabling it to exclude infeasible tasks and reset its task priorities. Finally, individual transportation, cooperative transportation, and the temporary exclusion of tasks considered infeasible are achieved. The scalability and versatility of the proposed method were confirmed through numerical experiments with an increased number of robots and objects, including unlearned weight objects. The effectiveness of the temporary deadlock avoidance was also confirmed by introducing additional robots within an episode. The proposed method enables the implementation of task allocation strategies that are feasible for different numbers of robots and various transport tasks without prior consideration of feasibility.

Problem

• Allocates tasks for multi-object transport by robots
• Addresses challenges with unknown object weights
• Prevents deadlock from untransportable objects

Innovation

• Reinforcement Learning task allocation
• Dynamic exclusion of infeasible tasks
• Scalable multi-robot cooperation framework