Value bounds and Convergence Analysis for Averages of LRP attributions

2025-09-10

AI Summary

This work investigates the numerical stability and convergence properties of Layer-wise Relevance Propagation (LRP)-based attribution methods. Addressing the mean convergence of attributions—particularly for LRP-beta—under multiple non-geometric data augmentations and SmoothGrad variants, we propose a corrected gradient matrix product modeling framework. Leveraging analogies to the Jacobian and singular value upper-bound analysis, we derive, for the first time, component-wise attribution bounds that are independent of network weight norms. Theoretically, we prove that LRP-beta exhibits stable convergence under repeated non-geometric augmentations, with convergence constants independent of weight norm—marking a significant improvement over gradient-based methods and LRP-epsilon. Extensive experiments quantitatively validate the theory’s predictive power for attribution stability and reliability. This establishes the first convergence theory for LRP tailored to non-geometric augmentation scenarios, advancing the theoretical foundations of explainable AI.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

We analyze numerical properties of Layer-wise relevance propagation (LRP)-type attribution methods by representing them as a product of modified gradient matrices. This representation creates an analogy to matrix multiplications of Jacobi-matrices which arise from the chain rule of differentiation. In order to shed light on the distribution of attribution values, we derive upper bounds for singular values. Furthermore we derive component-wise bounds for attribution map values. As a main result, we apply these component-wise bounds to obtain multiplicative constants. These constants govern the convergence of empirical means of attributions to expectations of attribution maps. This finding has important implications for scenarios where multiple non-geometric data augmentations are applied to individual test samples, as well as for Smoothgrad-type attribution methods. In particular, our analysis reveals that the constants for LRP-beta remain independent of weight norms, a significant distinction from both gradient-based methods and LRP-epsilon.

Problem

• Analyzing numerical properties of LRP attribution methods
• Deriving bounds for singular values and attribution values
• Establishing convergence constants for empirical attribution means

Innovation

• Matrix product representation for LRP attributions
• Upper bounds for singular values derived
• Convergence constants independent of weight norms