SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches

2026-06-16

AI Summary

This work addresses the limited semantic clarity and intuitiveness of existing saliency map explanations, which often fail to bridge the interpretability gap between model decisions and human understanding. To overcome this, we propose SketchXplain—the first framework to integrate artistic sketching into explainable AI. By synergistically combining salient region selection, knowledge alignment via concept bottleneck models, and a sketch optimization algorithm, SketchXplain generates visual explanations that are concise, semantically coherent, and cognitively aligned with human perception. User studies on facial expression recognition and skin lesion diagnosis demonstrate that, compared to conventional saliency maps or naive drawings, SketchXplain significantly enhances explanation comprehensibility and provides stronger support for clinical decision-making.

Abstract

Saliency map visualizations explain image-based AI predictions by pointing to regions, but these are often unintuitive and semantically unclear, leaving an interpretability gap. We argue that AI explanations should be intuitive -- coherent to user knowledge, yet simple and selective to accelerate interpretation. Inspired by artistic drawings, we propose SketchXplain to generate sketch-based visual explanations for intuitive image-based explainable AI (XAI). Combining techniques in saliency maps, concept-bottleneck models, and sketch optimization, SketchXplain integrates saliency to select coherent observation artifacts, concepts for knowledge coherence, cues to represent them, and abstraction for simplicity. Evaluating on face expression recognition, modeling and user studies showed that SketchXplain supported quicker interpretation with more aligned visualizations than saliency maps or simple drawings. Further evaluation on skin lesion diagnosis found that SketchXplain more coherently visualized disease symptoms, better supporting lay diagnosis. Thus, this work illustrates the value of sketches for intuitive, simple, coherent, and quick image-based XAI visualizations.

Problem

• saliency maps
• explainable AI
• visual explanations
• interpretability gap
• image classifiers

Innovation

• sketch-based explanation
• explainable AI
• saliency maps
• concept bottleneck
• visual abstraction