Segmentation and Smoothing Affect Explanation Quality More Than the Choice of Perturbation-based XAI Method for Image Explanations

2024-09-06

AI Summary

This study systematically investigates how design choices in perturbation-based image explanation methods—using RISE as a baseline—affect explanation quality. We conduct ablation studies across multiple datasets using proxy metrics (ROAR and KAR) to evaluate key components: patch-sampling strategies, image segmentation algorithms (e.g., SLIC, Watershed), smoothing techniques (e.g., Gaussian kernels), attribution computation schemes, and attribution granularity (pixel-level vs. region-level). Results demonstrate that segmentation quality and smoothing strength dominate explanation fidelity—exerting significantly greater influence than the choice of attribution formula. Moreover, pixel-level attribution consistently outperforms region-level attribution in discriminative power. The work identifies previously underappreciated yet critical design factors, providing empirically grounded guidance for explainable AI. To ensure reproducibility and facilitate further research, we publicly release all code and experimental data.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Perturbation-based post-hoc image explanation methods are commonly used to explain image prediction models. These methods perturb parts of the input to measure how those parts affect the output. Since the methods only require the input and output they can be applied to any model, making them a popular choice to explain black-box models. While many different models exist and have been compared with one another, it remains poorly understood which parameters of the different methods are responsible for their varying performance. This work uses the Randomized Input Sampling for Explanations (RISE) method as a baseline to evaluate many combinations of mask sampling, segmentation techniques, smoothing, attribution calculation, and per-segment or per-pixel attribution, using a proxy metric. The results show that attribution calculation, which is frequently the focus of other works, has little impact on the results. Conversely, segmentation and per-pixel attribution, rarely examined parameters, have a significant impact. The implementation of and data gathered in this work are available online: https://github.com/guspih/post-hoc-image-perturbation and https://bit.ly/smooth-mask-perturbation

Problem

• Image Interpretability
• Explanation Methods
• Prediction Quality

Innovation

• Image Segmentation
• Pixel Influence
• Reproducibility