Learning Quantifiable Visual Explanations Without Ground-Truth

2026-05-18

AI Summary

This work addresses the challenge of objectively evaluating explainable artificial intelligence (XAI) methods in the absence of ground-truth explanation labels. The authors propose a novel, label-free framework for quantifiable evaluation and training of visual explanations. By formalizing the sufficiency and necessity of explanations through continuous input perturbations, they introduce, for the first time, a differentiable measure of explanation quality that serves as a supervisory signal to train a lightweight adapter module capable of generating causal visual explanations. The approach is model-agnostic, requiring no access to the internal architecture of the original black-box model and preserving its predictive performance. Experimental results demonstrate that the generated explanations consistently outperform existing methods across multiple quantitative metrics and align more closely with human judgments of explanation quality.

Abstract

Explainable AI (XAI) techniques are increasingly important for the validation and responsible use of modern deep learning models, but are difficult to evaluate due to the lack of good ground-truth to compare against. We propose a framework that serves as a quantifiable metric for the quality of XAI methods, based on continuous input perturbation. Our metric formally considers the sufficiency and necessity of the attributed information to the model's decision-making, and we illustrate a range of cases where it aligns better with human intuitions of explanation quality than do existing metrics. To exploit the properties of this metric, we also propose a novel XAI method, considering the case where we fine-tune a model using a differentiable approximation of the metric as a supervision signal. The result is an adapter module that can be trained on top of any black-box model to output causal explanations of the model's decision process, without degrading model performance. We show that the explanations generated by this method outperform those of competing XAI techniques according to a number of quantifiable metrics.

Problem

• Explainable AI
• ground-truth
• explanation quality
• evaluation metric
• visual explanations

Innovation

• quantifiable explanation
• input perturbation
• causal explanation
• differentiable metric
• model-agnostic XAI