explanation fidelity measurement

Current machine learning evaluation practices predominantly rely on surface-level performance metrics, often neglecting the internal mechanisms of models. This work proposes trustworthy interpretability as a central evaluation paradigm and, for the first time, systematically demonstrates that it satisfies core criteria from the philosophy of science—namely falsifiability, reproducibility, and predictive power. By constructing an evaluation framework that integrates causal analysis with mechanistic probing, the study delineates three functional pathways through which interpretability enables the identification of behavioral origins, detection of latent flaws, and prediction of potential failure modes. This approach advances model assessment beyond performance-oriented benchmarks toward a deeper understanding of underlying mechanisms.

Existing XAI evaluation lacks reliable, principled reference baselines for quantifying explanation quality. Method: This paper proposes Quality Gap Estimation (QGE), the first method to introduce “inverse explanations” as a conceptual quality reference—approximated via counterfactual perturbations and latent-space inverse mapping—to enable relative, comparable quantification of individual explanations across dimensions including faithfulness, localization, and robustness. QGE abandons conventional random baselines, instead grounding evaluation in semantically meaningful counterfactuals. Contribution/Results: QGE significantly improves statistical robustness and cross-model/cross-dataset transferability of explanation assessment. Experiments across diverse architectures and datasets show that QGE increases ranking consistency of explanation quality by 32% and reduces evaluation variance by 41% compared to random baselines, thereby enhancing the reliability of model behavior diagnosis.

This study investigates whether existing algorithmic evaluation metrics for counterfactual explanations align with users’ perceptions of explanation quality. Through user studies conducted on three datasets, the authors systematically compare widely used algorithmic metrics against multidimensional human subjective ratings of counterfactual explanations. Employing correlation analyses and multivariate regression models, they assess the consistency and predictive power of these metrics. The findings reveal that algorithmic metrics generally exhibit weak correlations with human judgments and are highly dataset-dependent. Moreover, increasing the number of metrics yields only marginal improvements in predictive performance. These results expose structural limitations in current evaluation practices, underscoring their inability to capture key aspects of explanation quality that matter to users, and provide empirical support for advancing human-centered evaluation paradigms in explainable AI.

Existing evaluation of explainable recommender systems over-relies on recommendation performance and subjective user feedback, lacking objective, content-oriented metrics for explanation veracity—the intrinsic informational quality of explanations. This paper introduces signal detection theory to explanation evaluation for the first time, proposing a dual-dimensional decomposition framework grounded in fidelity (explanation’s faithfulness to the underlying model) and attunement (explanation’s alignment with user expectations). Based on this, we develop a quantifiable Veracity scoring model that integrates decision-sensitivity analysis. Through multi-scenario simulation experiments, we demonstrate that the model effectively discriminates among explanations of varying informational quality, exhibiting strong discriminative power and robustness. Our work bridges a critical gap in objective, content-centric explanation assessment and establishes a novel, principled benchmark for developing and evaluating explainable recommender systems.

Existing evaluation metrics for explainable AI (XAI) lack theoretical grounding and fail to reliably quantify explanation quality in high-stakes applications. Method: We propose the first spectral-analysis-based, structured framework for explanation quality assessment. By performing spectral decomposition on explanation result matrices, we mathematically decouple two orthogonal, intrinsic dimensions—stability (robustness to input perturbations) and target sensitivity (responsiveness to prediction targets)—establishing an interpretable, quantifiable 2D quality benchmark. Our approach integrates pixel-flipping experiments, information entropy analysis, and multi-scale validation across MNIST and ImageNet. Results: Empirical evaluation reveals that mainstream metrics systematically overlook the inherent trade-off between stability and target sensitivity. Our framework provides the first theoretically principled foundation for XAI evaluation, enabling rigorous comparison, diagnosis, and optimization of explanation methods.

This work addresses the lack of a unified, multidimensional standard for evaluating explainability, which hinders reliable comparisons across models, datasets, and explainable AI (XAI) methods. The authors propose a comprehensive evaluation framework encompassing fidelity, conciseness, and stability, integrating diverse metrics into a unified explainability scoring system for the first time. Through systematic benchmarking of mainstream XAI approaches—including LIME and SHAP—on multiple open-source datasets, they construct an explainability meta-knowledge base. This knowledge base enables predictive scoring of explainability for new models and data contexts, significantly enhancing contextual adaptability. Experimental results validate the framework’s effectiveness and uncover patterns in how explainability varies with model architecture, data characteristics, and user backgrounds, offering a practical and scalable tool for trustworthy AI development.

This study addresses a critical gap in explainable AI evaluation by proposing a novel metric that quantifies the consistency of model explanations across inputs sharing the same label or subjected to label-preserving perturbations. Specifically, the method measures the cosine similarity of SHAP values among samples with identical labels, thereby capturing explanation stability under semantic-preserving variations. By integrating both class-wise consistency and robustness to minor input perturbations into a unified evaluation framework, this work pioneers a systematic approach to assessing cross-sample explanation coherence. Experiments on SST-2 and IMDB datasets using BERT, RoBERTa, and DistilBERT demonstrate that the proposed metric effectively identifies inconsistent explanatory behaviors—such as undue reliance on specific features—and offers superior diagnostic capability compared to conventional fidelity-based metrics, ultimately contributing to the development of more trustworthy AI systems.

This study addresses the prevalent ambiguity, inconsistency, and incompleteness in articulating explainability requirements for AI systems due to a lack of standardized specifications. Through a structured literature review and interviews with developers, the authors identify a set of explainability quality attributes, which are then refined via a large-scale survey of practitioners into ten core attributes. For the first time, these attributes are translated into a prioritized, actionable guideline for writing explainability requirements. Building on this foundation, the authors design a lightweight, iterative requirements engineering workflow augmented by a large language model to assist in requirement generation. An accompanying web-based tool reduces average requirement drafting time by 23.5%, and user evaluations indicate that the generated requirements match or slightly exceed manually written ones in terms of implementability and textual quality.

This study addresses the challenge that existing algorithmic explanations are often poorly understood and misapplied by non-expert users due to semantic ambiguity and insufficient contextual information, leading to a disconnect between explanations and actual decision-making. To bridge this gap, the authors propose an “Explanation Card” framework that augments widely used interpretability methods—such as SHAP and counterfactual explanations—with structured metadata specifying their applicability boundaries, robustness properties, and user-oriented interpretation guidance. By shifting explanatory responsibility from end users to explanation providers, this approach enhances the practical utility and regulatory compliance of model explanations, aligning with the transparency requirements of the EU AI Act. Empirical evaluations demonstrate that Explanation Cards significantly improve users’ comprehension accuracy of complex model explanations and effectively flag scenarios where explanations are unreliable, thereby facilitating trustworthy real-world deployment of algorithmic systems.

This study addresses the longstanding challenge of treating machine learning interpretability as a non-functional requirement lacking quantifiable metrics and validation mechanisms. To bridge this gap, the work proposes an innovative approach that reframes interpretability as a verifiable functional requirement through the integration of data and model provenance. By synergizing principles from requirements engineering and machine learning engineering, the authors develop a systematic and operational verification framework. This framework enables, for the first time, the explicit specification and empirical validation of interpretability requirements, thereby substantially enhancing the engineering rigor and trustworthiness of machine learning system development.