COIN: Collaborative Interaction-Aware Multi-Agent Reinforcement Learning for Self-Driving Systems

2026-03-25

AI Summary

This work addresses the challenge of achieving efficient and safe collaboration in dense, dynamic multi-agent autonomous driving scenarios. The authors propose COIN, a collaborative interaction-aware framework operating under the centralized training with decentralized execution (CTDE) paradigm, which jointly optimizes individual navigation objectives and global coordination goals. Key innovations include a two-level interaction-aware centralized critic architecture that integrates local pairwise interactions with global system dependencies, and a counterfactual individual-global twin-delayed deep deterministic policy gradient algorithm (CIG-TD3) to enhance credit assignment and policy coordination. Experimental results demonstrate that COIN significantly outperforms existing methods in both high-density urban traffic simulations and real-world robotic platforms, achieving state-of-the-art performance in safety and traffic efficiency.

Technology Category

• Multiagent Systems: Coordination and Collaboration
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Game Theory and Economic Paradigms: Coordination and Collaboration

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multi-Agent Self-Driving (MASD) systems provide an effective solution for coordinating autonomous vehicles to reduce congestion and enhance both safety and operational efficiency in future intelligent transportation systems. Multi-Agent Reinforcement Learning (MARL) has emerged as a promising approach for developing advanced end-to-end MASD systems. However, achieving efficient and safe collaboration in dynamic MASD systems remains a significant challenge in dense scenarios with complex agent interactions. To address this challenge, we propose a novel collaborative(CO-) interaction-aware(-IN) MARL framework, named COIN. Specifically, we develop a new counterfactual individual-global twin delayed deep deterministic policy gradient (CIG-TD3) algorithm, crafted in a "centralized training, decentralized execution" (CTDE) manner, which aims to jointly optimize the individual objectives (navigation) and the global objectives (collaboration) of agents. We further introduce a dual-level interaction-aware centralized critic architecture that captures both local pairwise interactions and global system-level dependencies, enabling more accurate global value estimation and improved credit assignment for collaborative policy learning. We conduct extensive simulation experiments in dense urban traffic environments, which demonstrate that COIN consistently outperforms other advanced baseline methods in both safety and efficiency across various system sizes. These results highlight its superiority in complex and dynamic MASD scenarios, as further validated through real-world robot demonstrations. Supplementary videos are available at https://marmotlab.github.io/COIN/

Problem

• Multi-Agent Self-Driving
• Collaboration
• Complex Agent Interactions
• Safety and Efficiency
• Dense Traffic Scenarios

Innovation

• multi-agent reinforcement learning
• interaction-aware
• centralized critic
• credit assignment
• autonomous driving