Learning to Interact in World Latent for Team Coordination

2025-09-29

AI Summary

In multi-agent reinforcement learning (MARL), partial observability leads to incomplete information, while explicit communication suffers from high overhead and poor robustness. To address these challenges, this paper proposes Interactive World Latent Representations (IWoL). IWoL learns a shared, task-relevant latent space that implicitly encodes inter-agent relationships and environment dynamics, enabling efficient decentralized execution and implicit coordination; it also supports optional explicit communication via a unified interface. By jointly optimizing representation learning and implicit communication, IWoL uses the same latent encoding for both state representation and inter-agent information exchange. Evaluated on four standard MARL benchmarks—including StarCraft II and Multi-Agent MuJoCo—IWoL consistently outperforms state-of-the-art methods, demonstrating superior generalizability, adaptability to diverse tasks, and robustness under partial observability and communication constraints.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This work presents a novel representation learning framework, interactive world latent (IWoL), to facilitate team coordination in multi-agent reinforcement learning (MARL). Building effective representation for team coordination is a challenging problem, due to the intricate dynamics emerging from multi-agent interaction and incomplete information induced by local observations. Our key insight is to construct a learnable representation space that jointly captures inter-agent relations and task-specific world information by directly modeling communication protocols. This representation, we maintain fully decentralized execution with implicit coordination, all while avoiding the inherent drawbacks of explicit message passing, e.g., slower decision-making, vulnerability to malicious attackers, and sensitivity to bandwidth constraints. In practice, our representation can be used not only as an implicit latent for each agent, but also as an explicit message for communication. Across four challenging MARL benchmarks, we evaluate both variants and show that IWoL provides a simple yet powerful key for team coordination. Moreover, we demonstrate that our representation can be combined with existing MARL algorithms to further enhance their performance.

Problem

• Learning decentralized multi-agent coordination without explicit communication
• Building representations capturing agent relations and world dynamics
• Overcoming limitations of message passing in partial observability

Innovation

• Learning interactive world latent for team coordination
• Constructing representation space capturing agent relations
• Enabling decentralized execution with implicit coordination