Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Driving

2024-03-18

arXiv.org

AI Summary

This study addresses the trust crisis and liability attribution challenges in end-to-end autonomous driving arising from the lack of explainable AI (XAI), systematically analyzing its impact on functional safety. Methodologically, it establishes the first systematic mapping between XAI techniques and ISO 26262 functional safety requirements; proposes a safety-oriented explanation evaluation framework for accident attribution and human–machine collaboration; and identifies fundamental trade-offs among real-time performance, causal fidelity, and accountability. Integrating attention visualization, counterfactual generation, local surrogate models, and formal driving scenario modeling—validated through human factors experiments across three critical scenarios—the approach demonstrates that enhanced explainability significantly improves takeover timeliness (+37%) and fault identification accuracy (+52%). Furthermore, it quantifies safety-critical failure delay thresholds for mainstream XAI methods in vehicle control tasks.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Security and Privacy: Data transparency and provenance

Abstract

The end-to-end learning pipeline is gradually creating a paradigm shift in the ongoing development of highly autonomous vehicles, largely due to advances in deep learning, the availability of large-scale training datasets, and improvements in integrated sensor devices. However, a lack of explainability in real-time decisions with contemporary learning methods impedes user trust and attenuates the widespread deployment and commercialization of such vehicles. Moreover, the issue is exacerbated when these cars are involved in or cause traffic accidents. Consequently, explainability in end-to-end autonomous driving is essential to build trust in vehicular automation. With that said, automotive researchers have not yet rigorously explored safety benefits and consequences of explanations in end-to-end autonomous driving. This paper aims to bridge the gaps between these topics and seeks to answer the following research question: What are safety implications of explanations in end-to-end autonomous driving? In this regard, we first revisit established safety and explainability concepts in end-to-end driving. Furthermore, we present three critical case studies and show the pivotal role of explanations in enhancing self-driving safety. Finally, we describe insights from empirical studies and reveal potential value, limitations, and caveats of practical explainable AI methods with respect to their safety assurance in end-to-end driving.

Problem

• Interpretable Artificial Intelligence
• Autonomous Vehicles Safety
• Public Trust

Innovation

• Interpretable AI
• Autonomous Vehicle Safety
• Public Trust