🤖 AI Summary
This study addresses the challenge of collision risk assessment for autonomous driving systems (ADS) by introducing the “scored collision” paradigm—a decoupled, probabilistic quantification of both bodily injury and property damage risks. Methodologically, it constructs counterfactual simulation scenarios from real-world driving data, integrating multi-agent conflict detection, agent role identification, response-point localization, probabilistic human driver behavior modeling, counterfactual trajectory sampling, and physics-informed collision severity estimation. Its key contributions are the first fair, apples-to-apples risk comparison between ADS and human drivers, and native support for multi-source uncertainty fusion. Evaluated on the SHRP2/Nexar dataset, the method achieves 99% prediction accuracy. Empirical evaluation shows that a specific ADS version reduces natural collisions by 4× and lowers scored collision risk by 62%; over 250,000 miles of testing, 96% of ADS-initiated conflicts exhibit lower risk than the human baseline.
📝 Abstract
We present a methodology for estimating collision risk from counterfactual simulated scenarios built on sensor data from automated driving systems (ADS) or naturalistic driving databases. Two-agent conflicts are assessed by detecting and classifying conflict type, identifying the agents' roles (initiator or responder), identifying the point of reaction of the responder, and modeling their human behavioral expectations as probabilistic counterfactual trajectories. The states are used to compute velocity differentials at collision, which when combined with crash models, estimates severity of loss in terms of probabilistic injury or property damage, henceforth called fractional collisions. The probabilistic models may also be extended to include other uncertainties associated with the simulation, features, and agents. We verify the effectiveness of the methodology in a synthetic simulation environment using reconstructed trajectories from 300+ collision and near-collision scenes sourced from VTTI's SHRP2 database and Nexar dashboard camera data. Our methodology predicted fractional collisions within 1% of ground truth collisions. We then evaluate agent-initiated collision risk of an arbitrary ADS software release by replacing the naturalistic responder in these synthetic reconstructions with an ADS simulator and comparing the outcome to human-response outcomes. Our ADS reduced naturalistic collisions by 4x and fractional collision risk by ~62%. The framework's utility is also demonstrated on 250k miles of proprietary, open-loop sensor data collected on ADS test vehicles, re-simulated with an arbitrary ADS software release. The ADS initiated conflicts that caused 0.4 injury-causing and 1.7 property-damaging fractional collisions, and the ADS improved collision risk in 96% of the agent-initiated conflicts.