Fault-Tolerant MARL for CAVs under Observation Perturbations for Highway On-Ramp Merging

2025-11-28

AI Summary

To address the insufficient robustness of multi-agent reinforcement learning (MARL) policies for connected and autonomous vehicles (CAVs) in highway on-ramp merging scenarios under perception data perturbations, this paper proposes a dual-agent collaborative fault-tolerant framework. First, an adversarial fault injection mechanism is introduced to enhance policy resilience against observation disturbances. Second, a self-diagnostic observation reconstruction module—capable of modeling spatiotemporal correlations—is designed to enable anomaly detection and recovery of trustworthy vehicle states. The method integrates MARL, adversarial training, sequential modeling, and state reconstruction, explicitly encoding inter-vehicle spatiotemporal dependencies. Experiments demonstrate that the framework maintains near-optimal safety and traffic efficiency across diverse observation failure modes—including sensor noise, occlusion, and communication dropout—significantly outperforming baseline MARL approaches while preserving robust decision-making under uncertainty.

Technology Category

• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Multi-Agent Reinforcement Learning (MARL) holds significant promise for enabling cooperative driving among Connected and Automated Vehicles (CAVs). However, its practical application is hindered by a critical limitation, i.e., insufficient fault tolerance against observational faults. Such faults, which appear as perturbations in the vehicles' perceived data, can substantially compromise the performance of MARL-based driving systems. Addressing this problem presents two primary challenges. One is to generate adversarial perturbations that effectively stress the policy during training, and the other is to equip vehicles with the capability to mitigate the impact of corrupted observations. To overcome the challenges, we propose a fault-tolerant MARL method for cooperative on-ramp vehicles incorporating two key agents. First, an adversarial fault injection agent is co-trained to generate perturbations that actively challenge and harden the vehicle policies. Second, we design a novel fault-tolerant vehicle agent equipped with a self-diagnosis capability, which leverages the inherent spatio-temporal correlations in vehicle state sequences to detect faults and reconstruct credible observations, thereby shielding the policy from misleading inputs. Experiments in a simulated highway merging scenario demonstrate that our method significantly outperforms baseline MARL approaches, achieving near-fault-free levels of safety and efficiency under various observation fault patterns.

Problem

• Addresses insufficient fault tolerance in MARL for CAV observation perturbations
• Generates adversarial perturbations to stress policies during training
• Equips vehicles with self-diagnosis to detect faults and reconstruct observations

Innovation

• Adversarial fault injection agent co-trained for robustness
• Self-diagnosis mechanism using spatio-temporal correlations
• Reconstructing credible observations from corrupted vehicle data