TACTIC: Task-Agnostic Contrastive pre-Training for Inter-Agent Communication

2025-01-04

AI Summary

In multi-agent reinforcement learning, the “field-of-view (FoV) dilemma”—where cooperative performance collapses under extreme FoV settings (either extremely narrow or wide)—hinders generalization across heterogeneous observation ranges. To address this, we propose a task-agnostic contrastive communication pretraining framework. Our method jointly encodes local observations and communication messages via a message encoding-fusion architecture, and aligns message representations with global state embeddings through contrastive learning—enabling FoV-invariant communication without FoV labels or downstream fine-tuning. Evaluated on multiple SMACv2 scenarios, our approach consistently outperforms state-of-the-art methods, achieving 15–32% higher win rates under extreme FoV transfer. Moreover, it significantly improves communication robustness and cross-FoV generalization, demonstrating strong scalability to unseen observation ranges.

Technology Category

• Multiagent Systems: Agent Communication
• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The"sight range dilemma"in cooperative Multi-Agent Reinforcement Learning (MARL) presents a significant challenge: limited observability hinders team coordination, while extensive sight ranges lead to distracted attention and reduced performance. While communication can potentially address this issue, existing methods often struggle to generalize across different sight ranges, limiting their effectiveness. We propose TACTIC, Task-Agnostic Contrastive pre-Training strategy Inter-Agent Communication. TACTIC is an adaptive communication mechanism that enhances agent coordination even when the sight range during execution is vastly different from that during training. The communication mechanism encodes messages and integrates them with local observations, generating representations grounded in the global state using contrastive learning. By learning to generate and interpret messages that capture important information about the whole environment, TACTIC enables agents to effectively"see"more through communication, regardless of their sight ranges. We comprehensively evaluate TACTIC on the SMACv2 benchmark across various scenarios with broad sight ranges. The results demonstrate that TACTIC consistently outperforms traditional state-of-the-art MARL techniques with and without communication, in terms of generalizing to sight ranges different from those seen in training, particularly in cases of extremely limited or extensive observability.

Problem

• Multi-Agent Learning
• Field of View Dilemma
• Collaboration Efficiency

Innovation

• TACTIC
• Multi-agent Communication
• Adaptive Field of View