🤖 AI Summary
Current evaluation of visual explanation methods is hindered by scarce human-annotated ground-truth explanations and the absence of standardized, comprehensive evaluation protocols—limiting simultaneous assessment of alignment (faithfulness to model behavior) and causality (counterfactual robustness). To address this, we introduce the first standardized benchmark for explainability in image classification: it comprises eight cross-domain datasets, each with human-annotated explanations. We propose a unified evaluation framework that systematically integrates six quantitative metrics—including Infidelity, ROAR, and Faithfulness—to enable fair, comparable assessment of both post-hoc and intrinsic explanation methods. Furthermore, we release an open-source, multi-dataset, multi-metric evaluation platform. Empirical evaluation across four datasets and eight state-of-the-art methods demonstrates the benchmark’s discriminative power and robustness. All code and datasets are publicly available.
📝 Abstract
The rise of deep learning has ushered in significant progress in computer vision (CV) tasks, yet the"black box"nature of these models often precludes interpretability. This challenge has spurred the development of Explainable Artificial Intelligence (XAI) by generating explanations to AI's decision-making process. An explanation is aimed to not only faithfully reflect the true reasoning process (i.e., faithfulness) but also align with humans' reasoning (i.e., alignment). Within XAI, visual explanations employ visual cues to elucidate the reasoning behind machine learning models, particularly in image processing, by highlighting images' critical areas important to predictions. Despite the considerable body of research in visual explanations, standardized benchmarks for evaluating them are seriously underdeveloped. In particular, to evaluate alignment, existing works usually merely illustrate a few images' visual explanations, or hire some referees to report the explanation quality under ad-hoc questionnaires. However, this cannot achieve a standardized, quantitative, and comprehensive evaluation. To address this issue, we develop a benchmark for visual explanation, consisting of eight datasets with human explanation annotations from various domains, accommodating both post-hoc and intrinsic visual explanation methods. Additionally, we devise a visual explanation pipeline that includes data loading, explanation generation, and method evaluation. Our proposed benchmarks facilitate a fair evaluation and comparison of visual explanation methods. Building on our curated collection of datasets, we benchmarked eight existing visual explanation methods and conducted a thorough comparison across four selected datasets using six alignment-based and causality-based metrics. Our benchmark will be accessible through our website https://xaidataset.github.io.
