Empowering Multi-Robot Cooperation via Sequential World Models

2025-09-16

AI Summary

This work addresses two key challenges in multi-robot coordination: the difficulty of joint dynamics modeling and low communication efficiency. To this end, we propose the Sequentialized World Model (SeqWM) framework, which decouples joint dynamics into temporally structured, agent-wise world models and introduces a sequential communication mechanism to enable explicit intent sharing and linear communication complexity—thereby supporting predictive adaptation and dynamic role assignment. Technically, SeqWM integrates latent-variable rollout modeling, agent-specific world model learning, and model-driven multi-agent reinforcement learning. Evaluated on Bi-DexHands and Multi-Quad simulation benchmarks, SeqWM significantly outperforms existing model-based RL approaches in both task performance and sample efficiency. Furthermore, it has been successfully deployed on a physical quadruped robot platform, demonstrating strong generalization capability and practical applicability.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Coordination and Collaboration
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Model-based reinforcement learning (MBRL) has shown significant potential in robotics due to its high sample efficiency and planning capability. However, extending MBRL to multi-robot cooperation remains challenging due to the complexity of joint dynamics. To address this, we propose the Sequential World Model (SeqWM), a novel framework that integrates the sequential paradigm into model-based multi-agent reinforcement learning. SeqWM employs independent, sequentially structured agent-wise world models to decompose complex joint dynamics. Latent rollouts and decision-making are performed through sequential communication, where each agent generates its future trajectory and plans its actions based on the predictions of its predecessors. This design enables explicit intention sharing, enhancing cooperative performance, and reduces communication overhead to linear complexity. Results in challenging simulated environments (Bi-DexHands and Multi-Quad) show that SeqWM outperforms existing state-of-the-art model-free and model-based baselines in both overall performance and sample efficiency, while exhibiting advanced cooperative behaviors such as predictive adaptation and role division. Furthermore, SeqWM has been success fully deployed on physical quadruped robots, demonstrating its effectiveness in real-world multi-robot systems. Demos and code are available at: https://github.com/zhaozijie2022/seqwm-marl

Problem

• Extending model-based reinforcement learning to multi-robot cooperation
• Reducing communication overhead in multi-agent systems
• Enhancing cooperative performance through explicit intention sharing

Innovation

• Sequential World Model for multi-agent reinforcement learning
• Independent agent-wise models decompose joint dynamics
• Sequential communication enables intention sharing with linear complexity