Decompose Sparsely Where You Should, Absorb Densely Where You Should No

2026-06-11

AI Summary

Standard sparse autoencoders assume that the residual stream is fully amenable to sparse decomposition, overlooking the potential presence of dense components that are computationally critical yet resistant to sparse representation. This work introduces the concept of a “computational scaffold”—a compact, structurally identifiable, and causally necessary dense structure within the residual stream that is redundantly encoded by sparse dictionaries. To isolate this component, we augment the sparse autoencoder with a low-rank linear bottleneck in parallel, which first extracts the dense signal before performing sparse reconstruction. Experiments on layer 12 of Gemma-2-2B demonstrate that a rank-24 bottleneck reduces dense latent variables by 84%, substantially improving sparse probing performance and confirming the functional importance of this dense component to the model’s operation.

Abstract

Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition. We question this assumption and hypothesize that activations contain a low-rank, dense component that is computationally important to the model yet inherently unsuitable for sparse representation, which serves as a major source of the persistent dense latents widely observed in trained SAEs. To test this, we add a small rank-$r$ linear bottleneck in parallel with standard SAEs (BatchTopK and Matryoshka), allowing dense structure to be absorbed before sparse reconstruction. On Gemma-2-2B layer 12, a rank-24 bottleneck reduces dense latent count by up to 84\% while improving sparse probing and targeted probe perturbation on both architectures at matched sparsity. The absorbed component is (i) \textbf{structurally identifiable} as the top principal components and outlier dimensions; (ii) \textbf{causally necessary}, with removing it raising next-token cross-entropy by 7.5$\times$, far exceeding the 2.8$\times$ from removing the geometrically near-identical top-24 PCA directions; and (iii) \textbf{redundantly encoded by sparse dictionaries}, with ablating 787 maximally aligned sparse features raising cross-entropy by only 2.9$\times$ and ablating 2,048 topic-aligned features leaving MMLU topic classification virtually unchanged, whereas removing the scaffold drops it from 98.7\% to chance. Together, our findings identify a compact, semantically informative and causally important component of residual stream activations (which we term a \textbf{computational scaffold}) that standard sparse dictionaries represent inefficiently, suggesting that the scope of sparsity-based interpretability methods warrants careful re-examination.

Problem

• sparse autoencoders
• residual stream
• dense component
• computational scaffold
• sparsity

Innovation

• sparse autoencoders
• computational scaffold
• low-rank bottleneck
• residual stream decomposition
• dense latents