Learning Multi-agent Multi-machine Tending by Mobile Robots

2024-08-29

arXiv.org

AI Summary

To address manufacturing labor shortages and the limited flexibility of conventional fixed-base single-arm robots, this paper proposes a mobile-robot-based multi-agent collaborative material handling system. Methodologically, we introduce AB-MAPPO—the first attention-enhanced variant of MAPPO—specifically designed to handle sparse observations and strong task interdependence in industrial environments. AB-MAPPO integrates distributed state modeling with a custom sparse reward function to enable dynamic scheduling and real-time cooperative decision-making. Experimental results demonstrate substantial improvements in task success rate, operational safety, and resource utilization across multi-robot coordination scenarios. Ablation studies confirm the critical contributions of both the attention mechanism and the tailored observation modeling. Overall, AB-MAPPO consistently outperforms standard MAPPO across all key metrics.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Robotics can help address the growing worker shortage challenge of the manufacturing industry. As such, machine tending is a task collaborative robots can tackle that can also highly boost productivity. Nevertheless, existing robotics systems deployed in that sector rely on a fixed single-arm setup, whereas mobile robots can provide more flexibility and scalability. In this work, we introduce a multi-agent multi-machine tending learning framework by mobile robots based on Multi-agent Reinforcement Learning (MARL) techniques with the design of a suitable observation and reward. Moreover, an attention-based encoding mechanism is developed and integrated into Multi-agent Proximal Policy Optimization (MAPPO) algorithm to boost its performance for machine tending scenarios. Our model (AB-MAPPO) outperformed MAPPO in this new challenging scenario in terms of task success, safety, and resources utilization. Furthermore, we provided an extensive ablation study to support our various design decisions.

Problem

• Address manufacturing worker shortage with robotics
• Enhance machine tending flexibility via mobile robots
• Improve multi-agent reinforcement learning for robot efficiency

Innovation

• Mobile robots enhance manufacturing flexibility.
• Multi-agent Reinforcement Learning optimizes machine tending.
• Attention-based encoding boosts MAPPO algorithm performance.