🤖 AI Summary
This work addresses the limited interpretability of internal representations in vision models, which hinders understanding of their decision-making mechanisms. For the first time, sparse autoencoders are applied to decompose intermediate features of ConvNeXt trained on the FGVC-Aircraft dataset. Through analyses of activated image patches, activation magnitudes, and class selectivity, the study reveals that multiple sparse features correspond to identifiable aircraft parts or semantic visual patterns. Ablation studies linking features to input space, together with quantitative class selectivity metrics, systematically demonstrate these features’ influence on classification confidence and decision boundaries. The investigation also highlights inherent limitations, including feature ambiguity and coarse spatial localization, offering critical insights into the trade-offs between sparsity, interpretability, and representational fidelity in deep visual models.
📝 Abstract
Vision models can achieve strong performance on classification tasks, but the internal representations supporting their predictions are often difficult to interpret. This work investigates whether sparse autoencoders can decompose intermediate representations of a vision model into interpretable features. We train a ConvNeXt classifier on the FGVC-Aircraft dataset, extract spatial activations from its final feature stage, and train a sparse autoencoder on these activations. The learned sparse features are analyzed using top-activating image patches, activation strength, and class selectivity. Qualitative visual inspection reveals that several features correspond to recognizable aircraft structures and visual patterns. We evaluate a subset of selected features using input-space and feature-space ablations, measuring how blurring image patches and suppressing sparse features affect class logits, classification margins, and prediction confidence. The results suggest that sparse autoencoders can reveal partially interpretable, class-relevant visual features associated with aircraft recognition, while also exposing limitations such as polysemanticity and coarse spatial localization.