Causal-Inspired Multi-Agent Decision-Making via Graph Reinforcement Learning

2025-07-30

AI Summary

To address the challenges of collaborative multi-vehicle decision-making—namely poor interpretability and limited generalization—in complex traffic scenarios, this paper proposes a novel framework integrating Causal Disentangled Representation Learning (CDRL) with Graph Reinforcement Learning (GRL). The method employs causal intervention to identify and disentangle critical causal factors governing interactive decisions—such as vehicle intent and right-of-way relationships—and embeds these disentangled representations into a graph neural network architecture to guide multi-agent policy learning. Compared to state-of-the-art approaches, the framework significantly improves safety and efficiency in highly dynamic environments, such as unsignalized intersections: experiments demonstrate a 32.7% reduction in collision rate and a 24.5% increase in cumulative reward. Moreover, it achieves superior cross-scenario generalization and enhanced decision interpretability through explicit causal feature decomposition.

Technology Category

• Machine Learning: Causal Learning
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Reasoning under Uncertainty: Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Since the advent of autonomous driving technology, it has experienced remarkable progress over the last decade. However, most existing research still struggles to address the challenges posed by environments where multiple vehicles have to interact seamlessly. This study aims to integrate causal learning with reinforcement learning-based methods by leveraging causal disentanglement representation learning (CDRL) to identify and extract causal features that influence optimal decision-making in autonomous vehicles. These features are then incorporated into graph neural network-based reinforcement learning algorithms to enhance decision-making in complex traffic scenarios. By using causal features as inputs, the proposed approach enables the optimization of vehicle behavior at an unsignalized intersection. Experimental results demonstrate that our proposed method achieves the highest average reward during training and our approach significantly outperforms other learning-based methods in several key metrics such as collision rate and average cumulative reward during testing. This study provides a promising direction for advancing multi-agent autonomous driving systems and make autonomous vehicles' navigation safer and more efficient in complex traffic environments.

Problem

• Enhance decision-making in multi-vehicle interactive environments
• Integrate causal learning with reinforcement learning for autonomous driving
• Optimize vehicle behavior at unsignalized intersections using causal features

Innovation

• Causal disentanglement representation learning for feature extraction
• Graph neural network-based reinforcement learning integration
• Optimizing vehicle behavior at unsignalized intersections