From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Models

2026-05-01

AI Summary

Current visual models lack a unified framework, leading to fragmented local, global, and mechanistic interpretability. This work proposes a unified interpretability framework centered on the instantiated effective receptive field (iERF), which leverages Shared Ratio Decomposition (SRD) to generate high-fidelity saliency maps, integrates Concept-Anchored Feature Explanation (CAFE) to link abstract features with pixel-level evidence, and employs Inter-layer Concept Attribution and Trajectory (ICAT) to uncover concept evolution pathways within deep networks. For the first time, this approach unifies the three interpretability paradigms, significantly outperforming baselines on ResNet50, VGG16, and ViT in both fidelity and robustness. It successfully dissects sparse autoencoder features and clearly delineates dominant concept trajectories underlying correct predictions, misclassifications, and adversarial examples.

Abstract

Modern vision models achieve remarkable accuracy, but explaining where evidence arises, what the model encodes, and how internal computations assemble that evidence remains fragmented. We introduce an iERF-centric framework that unifies local, global, and mechanistic interpretability around a single analysis unit: the pointwise feature vector (PFV) paired with its instance-specific Effective Receptive Field (iERF). On the local side, Sharing Ratio Decomposition (SRD) expresses each PFV as a mixture of upstream PFVs via sharing ratios and propagates iERFs to construct class-discriminative saliency maps. SRD yields high-resolution, activation-faithful explanations, is robust to targeted manipulation and noise, and remains activation-agnostic across common nonlinearities. For the global view, we introduce Concept-Anchored Feature Explanation (CAFE), which utilizes the iERF as a semantic label, grounding abstract latent vectors in verifiable pixel-level evidence. With CAFE, we address the challenge of non-localized sparse autoencoder latents--especially in Transformers, where early self-attention mixes distant context. To answer how representations are composed through depth, we propose the Interlayer Concept Graph with Interlayer Concept Attribution (ICAT), which quantifies concept-to-concept influence while isolating layer pairs; an interlayer insertion, deletion protocol identifies Integrated Gradients as the most faithful instantiation. Empirically, across ResNet50, VGG16, and ViTs, our framework outperforms baselines in both fidelity and robustness, successfully interprets dispersed SAE features, and exposes dominant concept routes in correct, misclassified, and adversarial cases. Grounded in iERFs, our approach provides a coherent, evidence-backed map from pixels to concepts to decisions.

Problem

• interpretability
• vision models
• Effective Receptive Field
• feature explanation
• concept attribution

Innovation

• iERF
• Sharing Ratio Decomposition
• Concept-Anchored Feature Explanation
• Interlayer Concept Attribution
• mechanistic interpretability