Assessing the Noise Robustness of Class Activation Maps: A Framework for Reliable Model Interpretability

2025-08-25

AI Summary

This study systematically evaluates the explanation robustness of Class Activation Mapping (CAM) under noise perturbations. Addressing the lack of quantitative standards for CAM stability, we propose a two-dimensional robustness evaluation framework that jointly measures consistency—i.e., spatial overlap of activation regions across perturbed inputs—and responsiveness—i.e., sensitivity to semantically meaningful perturbations. We conduct extensive noise-robustness experiments across multiple state-of-the-art CAM methods, model architectures (e.g., ResNet, ViT), and benchmark image datasets (e.g., ImageNet, CUB-200), incorporating Gaussian noise, salt-and-pepper noise, and adversarial perturbations, with rigorous statistical validation. Results reveal significant heterogeneity in CAM method sensitivity across noise types and dataset characteristics. Our proposed metrics effectively discriminate between methods in terms of explanation stability, enabling reliable, reproducible assessment of explanation robustness—thereby advancing trustworthy model interpretability.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Class Activation Maps (CAMs) are one of the important methods for visualizing regions used by deep learning models. Yet their robustness to different noise remains underexplored. In this work, we evaluate and report the resilience of various CAM methods for different noise perturbations across multiple architectures and datasets. By analyzing the influence of different noise types on CAM explanations, we assess the susceptibility to noise and the extent to which dataset characteristics may impact explanation stability. The findings highlight considerable variability in noise sensitivity for various CAMs. We propose a robustness metric for CAMs that captures two key properties: consistency and responsiveness. Consistency reflects the ability of CAMs to remain stable under input perturbations that do not alter the predicted class, while responsiveness measures the sensitivity of CAMs to changes in the prediction caused by such perturbations. The metric is evaluated empirically across models, different perturbations, and datasets along with complementary statistical tests to exemplify the applicability of our proposed approach.

Problem

• Evaluating noise robustness of Class Activation Maps
• Assessing CAM sensitivity to various noise perturbations
• Proposing a metric for CAM consistency and responsiveness

Innovation

• Proposed CAM robustness metric with consistency
• Evaluated multiple CAM methods under noise
• Assessed dataset impact on explanation stability