Interpretable Embeddings with Sparse Autoencoders: A Data Analysis Toolkit

2025-12-10

AI Summary

Current large-scale text analysis relies either on costly large language models (LLMs) or dense embeddings lacking semantic controllability, hindering precise detection of model biases and data biases. This paper introduces an interpretable embedding method based on sparse autoencoders (SAEs), wherein embedding dimensions are explicitly aligned with human-understandable semantic concepts. The approach enables bias detection, concept association discovery, controllable clustering, and attribute-based retrieval. We present the first systematic evaluation demonstrating SAEs’ integrated advantages in interpretability, controllability, and cost-efficiency—enabling concept-level intervention, cross-model behavioral attribution, and training-data trigger pattern mining. Compared to LLM-based methods, our approach reduces computational cost by 2–8× while significantly improving bias identification reliability. It consistently outperforms dense embeddings across four benchmark tasks. Empirically, we localize Grok-4’s ambiguity-resolution tendency and identify training-data-triggered phrases in Tulu-3.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Analyzing large-scale text corpora is a core challenge in machine learning, crucial for tasks like identifying undesirable model behaviors or biases in training data. Current methods often rely on costly LLM-based techniques (e.g. annotating dataset differences) or dense embedding models (e.g. for clustering), which lack control over the properties of interest. We propose using sparse autoencoders (SAEs) to create SAE embeddings: representations whose dimensions map to interpretable concepts. Through four data analysis tasks, we show that SAE embeddings are more cost-effective and reliable than LLMs and more controllable than dense embeddings. Using the large hypothesis space of SAEs, we can uncover insights such as (1) semantic differences between datasets and (2) unexpected concept correlations in documents. For instance, by comparing model responses, we find that Grok-4 clarifies ambiguities more often than nine other frontier models. Relative to LLMs, SAE embeddings uncover bigger differences at 2-8x lower cost and identify biases more reliably. Additionally, SAE embeddings are controllable: by filtering concepts, we can (3) cluster documents along axes of interest and (4) outperform dense embeddings on property-based retrieval. Using SAE embeddings, we study model behavior with two case studies: investigating how OpenAI model behavior has changed over time and finding "trigger" phrases learned by Tulu-3 (Lambert et al., 2024) from its training data. These results position SAEs as a versatile tool for unstructured data analysis and highlight the neglected importance of interpreting models through their data.

Problem

• Developing cost-effective interpretable embeddings for text analysis
• Enhancing control over concept properties in embedding models
• Uncovering semantic differences and biases in large datasets

Innovation

• Sparse autoencoders create interpretable concept-based embeddings
• SAE embeddings are more cost-effective and reliable than LLMs
• SAE embeddings offer controllable concept filtering for document analysis