You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents

2026-05-26

AI Summary

This work addresses the challenge of emergent cooperation among unaligned agents in open, distributed multi-agent systems as population scales grow. The authors propose an "alignment propagation" mechanism that deploys a single aligned seed agent to guide untrained agents toward cooperative behavior through natural language interaction, reframing multi-agent alignment as a scalable social capability engineering problem. Leveraging cooperation reasoning and persuasion strategies distilled from the Qwen-3-14B model, the approach increases cooperation rates from 24.8% to 62.2% in the Red-Black Game and achieves zero-shot transfer to the Sugarscape environment, yielding a 91.5% transaction success rate—substantially outperforming the baseline of 21.6%. These results demonstrate the method’s effectiveness and strong generalization across distinct multi-agent settings.

Abstract

Ensuring agent behaviors in distributed open multi-agent systems remains challenging, especially as populations grow and unaligned agents may exist. We show that a single aligned agent can propagate cooperative behaviors to untrained agents purely through natural language interaction, a phenomenon we term Alignment Propagation. We study this in the Red-Black Game, a team-based iterated Prisoner's Dilemma in which teammates deliberate and vote to determine their team's collective action. By distilling the cooperative reasoning and persuasive dialogues of a teacher model into a Qwen-3-14B, we obtain a seed agent that, when placed among four untrained teammates, doubles the cooperation rate from 24.8% to 62.2%, outperforming the teacher model and a vanilla Gemini-3.1-Pro. Remarkably, a seed trained exclusively on the RedBlack Game transfers zero-shot to Sugarscape, a spatially grounded survival simulation with pairwise trading, achieving a 91.5% trade success rate versus a 21.6% baseline. Our results reframe multi-agent alignment from an exhaustive per-agent training problem to a scalable social capability that can be engineered through strategic seed placement.

Problem

• multi-agent systems
• alignment
• cooperative behaviors
• open environments
• behavior propagation

Innovation

• Alignment Propagation
• Seed Agent
• Multi-Agent Alignment
• Zero-Shot Transfer
• Cooperative Behavior