People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Explainable AI

2024-03-11

AI Summary

This study investigates how eXplainable AI (XAI) for autonomous driving can align with human cognitive patterns—specifically, how humans naturally interpret autonomous vehicle behavior—and comparatively evaluates teleological, mechanistic, and counterfactual explanation paradigms. Guided by cognitive theory, we conducted a two-stage human factors experiment across 14 representative driving scenarios, yielding HEADD—the first high-quality, human-annotated explanation dataset for autonomous driving. Empirical results show that teleological explanations significantly outperform the others in both comprehensibility and trustworthiness; perceived intentionality emerges as the strongest predictor of explanation quality. Crucially, explanation paradigm is established as a core analytical dimension for XAI design and evaluation. The work advances cognition-informed, empirical XAI research, and we publicly release HEADD alongside associated code, offering a novel paradigm for explainability modeling. (149 words)

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

It is often argued that effective human-centered explainable artificial intelligence (XAI) should resemble human reasoning. However, empirical investigations of how concepts from cognitive science can aid the design of XAI are lacking. Based on insights from cognitive science, we propose a framework of explanatory modes to analyze how people frame explanations, whether mechanistic, teleological, or counterfactual. Using the complex safety-critical domain of autonomous driving, we conduct an experiment consisting of two studies on (i) how people explain the behavior of a vehicle in 14 unique scenarios (N1=54) and (ii) how they perceive these explanations (N2=382), curating the novel Human Explanations for Autonomous Driving Decisions (HEADD) dataset. Our main finding is that participants deem teleological explanations significantly better quality than counterfactual ones, with perceived teleology being the best predictor of perceived quality. Based on our results, we argue that explanatory modes are an important axis of analysis when designing and evaluating XAI and highlight the need for a principled and empirically grounded understanding of the cognitive mechanisms of explanation. The HEADD dataset and our code are available at: https://datashare.ed.ac.uk/handle/10283/8930.

Problem

• Human-centered explainable AI
• Cognitive science insights
• Autonomous driving explanations

Innovation

• Teleological explanations improve XAI design
• Cognitive science aids explainable AI frameworks
• HEADD dataset supports autonomous driving research