Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

2026-06-23

AI Summary

This work addresses the lack of quantitative methods for evaluating semantic alignment between latent variables in sparse autoencoders (SAEs) and human-interpretable concepts. To this end, the authors propose a human-grounded evaluation framework that obviates user studies by leveraging synthetic datasets (synCUB and synCOCO), a many-to-one matching algorithm termed Fully-Binary Matching Pursuit (FBMP), and an alignment metric called TAPAScore based on attribute perturbations. The framework systematically quantifies correspondence between SAE features and human-annotated concepts. Experimental results demonstrate that SAEs with moderate dictionary sizes achieve optimal interpretability, while excessive overcompleteness degrades perturbation-based alignment. Furthermore, the proposed method reliably distinguishes between trained and untrained SAEs, establishing a robust quantitative benchmark for interpretability assessment.

Abstract

Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention-style evaluation in vision, we construct synCUB and synCOCO, synthetic benchmarks of paired images that differ in exactly one attribute. We introduce Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and consistently outperforms one-to-one baselines. For functional validation, we propose a Targeted Attribute Perturbation Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image-level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2 embeddings, we find that increased overcompleteness can reduce perturbation alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs. Code and datasets are available at https://github.com/JonasKlotz/sae-concept-eval.

Problem

• interpretability
• sparse autoencoders
• concept alignment
• semantic correspondence
• human-annotated concepts

Innovation

• Sparse Autoencoders
• Interpretability Evaluation
• Concept Alignment
• Attribute Perturbation
• Synthetic Benchmarks