Cooperative-Competitive Team Play of Real-World Craft Robots

2026-02-24

AI Summary

This work addresses the challenges of low training efficiency in multi-agent reinforcement learning (MARL) for real-world robotic systems and the difficulty of sim-to-real (Sim2Real) transfer. To bridge this gap, the authors develop an integrated system comprising a high-fidelity simulator, a distributed MARL framework, and a physical multi-robot platform. They further propose an Out-of-Distribution State Initialization (OODSI) mechanism that effectively mitigates the domain discrepancy between simulation and reality. By exposing agents to diverse and challenging initial states during training, OODSI significantly enhances policy generalization in real-world environments. The approach is validated on collaborative and competitive multi-robot tasks, demonstrating a 20% improvement in Sim2Real performance over baseline methods.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Multiagent Learning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing agents in recent years. However, the efficient training of collective robots using multi-agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-robot car competitive game and a cooperative task in real-world settings.

Problem

• multi-agent reinforcement learning
• sim-to-real transfer
• cooperative-competitive tasks
• real-world robotics
• policy training

Innovation

• multi-agent reinforcement learning
• sim-to-real transfer
• Out of Distribution State Initialization
• cooperative-competitive robotics
• distributed learning framework