A Distributional View for Visual Mechanistic Interpretability: KL-Minimal Soft-Constraint Principle

2026-05-17

AI Summary

This work addresses the limitations of existing visual interpretability methods, which often rely on heuristic strategies that compromise either perceptual plausibility or mechanistic faithfulness. To overcome this, the paper introduces a novel theoretical framework grounded in a distributional perspective, modeling how feature activations perturb the natural image distribution. It proposes a KL-minimization soft-constraint principle that simultaneously ensures mechanistic faithfulness and alignment with human perception. To operationalize this principle, the authors develop an energy-guided diffusion posterior sampling algorithm. Experiments on DINOv3 demonstrate the method’s superior theoretical grounding and empirical performance, while also uncovering statistical biases inherent in current interpretability approaches.

Abstract

Most current paradigms in visual mechanistic interpretability (MI) remain confined to interpreting internal units of the vision model via heuristic methods (e.g., top-$K$ activation retrieval or optimization with regularization). In this work, we establish a theoretical distributional view for visual MI, which models the influence of a feature activation on the natural image distribution, thereby formulating a Kullback-Leibler (KL)-minimal optimization problem to model the MI task. Under this framework, statistical biases are identified within previous MI paradigms, which reveal that they may either be perceptually uninterpretable to humans (i.e., deviate from the natural image distribution), or mechanistically unfaithful to the vision models (i.e., unable to activate model features). To resolve the biases under the distributional view, we propose a model with a KL-minimal soft-constraint principle for visual MI that theoretically balances interpretability and faithfulness. We realize this principle via energy-guided diffusion posterior sampling. Extensive experiments validate the theoretical soundness of the proposed distributional view and demonstrate the practical effectiveness of our paradigm on the DINOv3 vision model.

Problem

• visual mechanistic interpretability
• distributional view
• KL-minimal
• faithfulness
• natural image distribution

Innovation

• distributional view
• KL-minimal optimization
• visual mechanistic interpretability
• energy-guided diffusion
• soft-constraint principle