SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization

2026-08-13

AI Summary

Current sparse autoencoders (SAEs) yield features from large language models (LLMs) that lack efficient and deep natural language interpretations, as existing approaches rely on external behavioral observations, resulting in shallow and computationally expensive explanations. This work proposes directly injecting SAE decoder directions into LLM representations and applying lightweight fine-tuning to downstream layers, enabling the model to autonomously generate natural language explanations for these features. This approach is the first to produce generalizable and transferable explanations directly from decoder directions, facilitating feature interpretation across different SAE dictionaries and LLMs. Moreover, it supports composing or inverting multiple directions to reflect semantic changes. Experiments demonstrate that the model accurately explains unseen features and exhibits strong capabilities in semantic composition and inversion under multi-directional interventions.

Abstract

Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale. We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations. Our experiments show that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries, and, with a lightweight adapter, extends to SAE features from different LLMs. Intervention experiments show that injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.

Problem

• sparse autoencoder
• feature explanation
• large language model
• representation interpretation
• computational inefficiency

Innovation

• Sparse Autoencoder
• Representation Verbalization
• Feature Interpretability
• Large Language Models
• Decoder Direction Injection