TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving

2026-02-27

AI Summary

This work addresses the challenges of coordination instability and safety risks faced by autonomous vehicles in dense traffic under partial observability, where rapidly evolving interaction patterns complicate decision-making. To this end, the authors propose a communication-free, decentralized multi-agent reinforcement learning framework that dynamically constructs a time-varying directed priority graph based on trajectory weaving relationships. This graph locally induces Stackelberg subgames, implicitly establishing leader–follower roles without requiring global ordering. By integrating action prediction, action-conditional value learning, and a centralized training with decentralized execution (CTDE) architecture, the approach significantly enhances coordination stability and scalability. Experimental results across four dense traffic scenarios demonstrate that the method substantially reduces collision rates while maintaining high traffic efficiency and smooth control, outperforming existing multi-agent reinforcement learning approaches.

Technology Category

• Multiagent Systems: Multiagent Learning
• Reasoning under Uncertainty: Sequential Decision Making
• Planning, Routing, and Scheduling: Planning/Scheduling and Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Safe and efficient autonomous driving in dense traffic is fundamentally a decentralized multi-agent coordination problem, where interactions at conflict points such as merging and weaving must be resolved reliably under partial observability. With only local and incomplete cues, interaction patterns can change rapidly, often causing unstable behaviors such as oscillatory yielding or unsafe commitments. Existing multi-agent reinforcement learning (MARL) approaches either adopt synchronous decision-making, which exacerbate non-stationarity, or depend on centralized sequencing mechanisms that scale poorly as traffic density increases. To address these limitations, we propose Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, which extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play. Conditioned on this graph, TSC endogenously factorizes dense interactions into graph-local Stackelberg subgames and, under centralized training and decentralized execution (CTDE), learns a sequential coordination policy that anticipates leaders via action prediction and trains followers through action-conditioned value learning to approximate local best responses, improving training stability and safety in dense traffic. Experiments across four dense traffic scenarios show that TSC achieves superior performance over representative MARL baselines across key metrics, most notably reducing collisions while maintaining competitive traffic efficiency and control smoothness.

Problem

• multi-agent reinforcement learning
• interactive driving
• decentralized coordination
• partial observability
• dense traffic

Innovation

• Stackelberg coordination
• multi-agent reinforcement learning
• interactive driving
• priority graph
• decentralized execution