Collaborative Multi-Agent Testing for Emergent Failure Discovery in Autonomous Driving Systems

2026-07-07

AI Summary

Autonomous driving systems often suffer from sudden failures due to internal module defects and complex interactions among perception, planning, and control, which existing testing methods struggle to detect effectively. To address this challenge, this work proposes CREAD, a novel framework that unifies perturbation generation, behavioral consistency verification, and exploration within a collaborative multi-agent architecture. CREAD employs a shared blackboard and a coordinator mechanism to integrate perception-focused fuzzing, metamorphic validation, and coordinating agents, enabling targeted perturbations guided by perception while supporting extensibility to other modules. Evaluated in the HighwayEnv simulation environment, CREAD achieves a 2.1× higher failure discovery rate per hundred scenarios on average compared to single-agent baselines and demonstrates strong performance in roundabout scenarios as well.

Abstract

Autonomous Driving Systems (ADS) can fail because of faults within individual modules as well as from interactions across perception, planning, and control. Yet existing ADS testing research often treats key testing functions, such as perturbation generation, behavioural assessment, and test case selection and exploration, as loosely coupled steps rather than coordinated roles for discovering such failures. We present CREAD, a collaborative multi-agent testing framework for testing ADS that organises perturbation generation, behavioural validation, and search coordination through a shared blackboard and an orchestrator. In the current work-in-progress instantiation, the framework focuses on perception-oriented perturbation generation, while remaining extensible to other ADS modules, including planning and control. It currently comprises a Perception Fuzzer Agent, a Metamorphic Validator Agent, and an Orchestrator Agent. Respectively, they generate perturbations, assess behavioural consistency across related scenario pairs, and coordinate further exploration. Experiments in HighwayEnv simulator show that the collaborative configuration improves failure discovery in the highway environment and remains competitive in the roundabout setting. Across the two environments, it yields about 2.1x as many failures per 100 scenarios as the single-agent baseline on average, while gains over a non-collaborative two-agent baseline vary across environments. These results suggest that collaborative multi-agent testing is a promising research direction for emergent ADS behaviour discovery.

Problem

• Autonomous Driving Systems
• Emergent Failure Discovery
• Collaborative Testing
• Multi-Agent Testing
• Perception Perturbation

Innovation

• collaborative multi-agent testing
• autonomous driving systems
• perception fuzzing
• metamorphic validation
• emergent failure discovery