🤖 AI Summary
Existing analyses of sparse autoencoder (SAE) features predominantly rely on activation tokens or decoder weights, which struggle to capture higher-order co-occurrence structures among features. This work addresses this limitation by modeling SAE features as token co-occurrence graphs and introduces a frequency-binned Weisfeiler–Lehman graph kernel to measure their structural similarity. The proposed approach uncovers coherent relational patterns that remain invisible under conventional analytical perspectives. Applied to SAE features derived from GPT-2 Small, the method successfully clusters semantically consistent motifs—such as punctuation-dense sequences, linguistic scripts, and code templates—demonstrating robustness across varying hyperparameters and random seeds, thereby validating both its effectiveness and reliability.
📝 Abstract
Sparse autoencoders (SAEs) have become central to mechanistic interpretability, decomposing transformer activations into monosemantic features. Yet existing analyses characterise features almost exclusively through top-activating token lists or decoder weight vectors, leaving the higher-order co-occurrence structure shared across features largely unexamined. We introduce a graph-structured representation in which each SAE feature is modelled as a token co-occurrence graph: nodes are the tokens most frequent near strong activations, and edges connect pairs that co-occur within local context windows. A custom WL-style, frequency-binned graph kernel then provides a similarity measure over this structural space. Applied as a proof of concept to features from a large SAE trained on GPT-2 Small and probed with a synthetic mixed-domain corpus, our clustering recovers heuristic motif families (punctuation-heavy patterns, language and script clusters, and code-like templates) that are not recovered by clustering on decoder cosine similarity. A token-histogram baseline achieves higher overall purity, so the contribution of the graph view is complementary rather than dominant: it surfaces structural relationships that token-frequency and decoder-weight views alone do not capture. Cluster assignments are stable across graph-construction hyperparameters and random seeds.