Benchmarking XAI Explanations with Human-Aligned Evaluations

2024-11-04

arXiv.org

AI Summary

This study addresses the misalignment between XAI explanation quality and human perception, as well as the lack of reliable, human-centered evaluation benchmarks. To this end, we propose PASTA: the first large-scale, multi-dataset (e.g., COCO), human-centric XAI evaluation benchmark featuring both image-level and concept-level human annotations. We introduce a data-driven, preference-learning-based interpretability metric grounded in human judgments, enabling cross-modal explanation comparison. Through rigorous psychophysical experiments, crowdsourced annotation, and statistical modeling, we identify strong human preference for saliency maps and demonstrate weak correlation between prevailing numerical metrics (e.g., faithfulness, localization) and actual human judgments. PASTA establishes a reproducible, scalable, and human-aligned evaluation standard for XAI—shifting assessment paradigms from “machine-readable” to “human-trustworthy.”

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Computer Vision: Interpretability, Explainability, and Transparency
• Knowledge Representation and Reasoning: Preferences

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

In this paper, we introduce PASTA (Perceptual Assessment System for explanaTion of Artificial intelligence), a novel framework for a human-centric evaluation of XAI techniques in computer vision. Our first key contribution is a human evaluation of XAI explanations on four diverse datasets (COCO, Pascal Parts, Cats Dogs Cars, and MonumAI) which constitutes the first large-scale benchmark dataset for XAI, with annotations at both the image and concept levels. This dataset allows for robust evaluation and comparison across various XAI methods. Our second major contribution is a data-based metric for assessing the interpretability of explanations. It mimics human preferences, based on a database of human evaluations of explanations in the PASTA-dataset. With its dataset and metric, the PASTA framework provides consistent and reliable comparisons between XAI techniques, in a way that is scalable but still aligned with human evaluations. Additionally, our benchmark allows for comparisons between explanations across different modalities, an aspect previously unaddressed. Our findings indicate that humans tend to prefer saliency maps over other explanation types. Moreover, we provide evidence that human assessments show a low correlation with existing XAI metrics that are numerically simulated by probing the model.

Problem

• Evaluating XAI techniques using human-aligned perceptual assessments
• Creating large-scale benchmark for diverse explanation methods comparison
• Developing automated scoring system predicting human explanation preferences

Innovation

• Human-centric framework for XAI evaluation
• Large-scale benchmark dataset for XAI techniques
• Automated scoring method predicting human preferences