Adaptive Human-AI Coordination via Hierarchical Action Disentanglement

2026-05-22

AI Summary

This work addresses the challenge of limited robustness in human-agent collaboration, where agents often struggle to adapt to diverse and unknown partner behaviors. To this end, the authors propose the Intrinsic Action Disentanglement (IAD) framework, a deep hierarchical reinforcement learning approach that governs low-level action sequences through high-level latent skills. IAD introduces an intrinsic reward mechanism to explicitly disentangle action distributions across distinct skills, thereby establishing an interpretable mapping between high-level decisions and partner-specific behaviors. Empirical results demonstrate that IAD significantly outperforms strong baselines across multiple Overcooked-AI layouts and partner configurations—including simulated agents, human proxy models, and real human partners—achieving more flexible and reliable collaborative performance.

Abstract

Human-AI collaboration requires agents that can adapt to diverse partner behaviors and skill levels while remaining robust to unseen partners. Existing methods often collapse to a single dominant behavior or learn poorly aligned skills, limiting effective coordination. We propose Intrinsic Action Disentanglement (IAD), a deep hierarchical reinforcement learning (DHRL) framework that learns distinct, partner-aware low-level action sequences conditioned on high-level latent skills. IAD introduces an intrinsic reward that explicitly encourages disentangled action distributions of the agent's low-level policy across skills, yielding an interpretable mapping between high-level decisions and partner-specific behavioral responses. By capturing temporally extended interaction patterns, IAD enables flexible adaptation to heterogeneous partner dynamics under distributional shift. We evaluate IAD in the Overcooked-AI domain across multiple layouts and diverse partner settings, including unseen simulated partners, a human-proxy model trained on human-human gameplay, and real human partners. Results show that IAD consistently outperforms strong baselines and achieves more reliable, adaptive coordination across all settings.

Problem

• Human-AI collaboration
• partner adaptation
• behavioral coordination
• distributional shift
• skill alignment

Innovation

• Intrinsic Action Disentanglement
• Hierarchical Reinforcement Learning
• Human-AI Collaboration
• Skill Disentanglement
• Adaptive Coordination