Symmetry-Breaking Augmentations for Ad Hoc Teamwork

2024-02-15

arXiv.org

AI Summary

In dynamic human-AI collaboration, AI agents struggle to rapidly adapt to unknown human teammates’ strategies. Method: We propose Symmetry-Breaking Augmentation (SBA), a novel framework that actively flips strategy symmetries during training to enhance behavioral diversity and generalization. SBA integrates behavior-diversity regularization, explicit modeling of strategy symmetry, and multi-agent reinforcement learning. Crucially, we introduce “strategy-set symmetry dependence” — a general, quantifiable metric for assessing symmetry reliance across policy sets. Contribution/Results: Evaluated in the Hanabi benchmark, SBA significantly outperforms state-of-the-art ad hoc teamwork baselines. It demonstrates superior robustness and real-time alignment when collaborating with diverse human conventions—spanning varying skill levels, communication styles, and implicit protocols. Our approach establishes a new paradigm for generalizable human-AI coordination in non-stationary, open-team settings, advancing adaptability beyond fixed-agent assumptions.

Technology Category

• Humans and AI: Human-AI Collaboration / Human-AI Teaming
• Multiagent Systems: Teamwork
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Search and Retrieval-Augmented AI: Agentic search

Abstract

In dynamic collaborative settings, for artificial intelligence (AI) agents to better align with humans, they must adapt to novel teammates who utilise unforeseen strategies. While adaptation is often simple for humans, it can be challenging for AI agents. Our work introduces symmetry-breaking augmentations (SBA) as a novel approach to this challenge. By applying a symmetry-flipping operation to increase behavioural diversity among training teammates, SBA encourages agents to learn robust responses to unknown strategies, highlighting how social conventions impact human-AI alignment. We demonstrate this experimentally in two settings, showing that our approach outperforms previous ad hoc teamwork results in the challenging card game Hanabi. In addition, we propose a general metric for estimating symmetry dependency amongst a given set of policies. Our findings provide insights into how AI systems can better adapt to diverse human conventions and the core mechanics of alignment.

Problem

• AI agents struggle to adapt to novel human teammates' strategies
• Symmetry-breaking augmentations enhance behavioral diversity for robustness
• Improving human-AI alignment by addressing symmetry dependency in policies

Innovation

• Uses symmetry-breaking augmentations for adaptation
• Increases behavioural diversity in training
• Proposes metric for symmetry dependency estimation