The Linear Centroids Hypothesis: How Deep Network Features Represent Data

2026-04-13

AI Summary

Existing interpretability methods for deep networks, such as the Linear Representation Hypothesis (LRH), struggle to precisely localize features to specific neurons or layers, are susceptible to spurious features, and lack cross-component applicability. This work proposes the Linear Centroid Hypothesis (LCH), which defines features as linear directions spanned by functional behavior centroids within local regions of input space, using centroids rather than raw activations as feature carriers. LCH is compatible with existing LRH-based tools, enables cross-layer analysis, and substantially improves the accuracy of identifying genuine features. It successfully discovers directly interpretable circuits in GPT2-Large. Experiments further demonstrate that LCH yields sparser and more effective feature dictionaries on the DINO model, achieving superior downstream performance and confirming its validity and generalizability.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Representation Learning
• Computer Vision: Large Vision Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Identifying and understanding the features that a deep network (DN) extracts from its inputs to produce its outputs is a focal point of interpretability research. The Linear Representation Hypothesis (LRH) identifies features in terms of the linear directions formed by the inputs in a DN's latent space. However, the LRH is limited as it abstracts away from individual components (e.g., neurons and layers), is susceptible to identifying spurious features, and cannot be applied across sub-components (e.g., multiple layers). In this paper, we introduce the Linear Centroids Hypothesis (LCH) as a new framework for identifying the features of a DN. The LCH posits that features correspond to linear directions of centroids, which are vector summarizations of the functional behavior of a DN in a local region of its input space. Interpretability studies under the LCH can leverage existing LRH tools, such as sparse autoencoders, by applying them to the DN's centroids rather than to its latent activations. We demonstrate that doing so yields sparser feature dictionaries for DINO vision transformers, which also perform better on downstream tasks. The LCH also inspires novel approaches to interpretability; for example, LCH can readily identify circuits in GPT2-Large. For code to study the LCH https://github.com/ThomasWalker1/LinearCentroidsHypothesis .

Problem

• deep network interpretability
• feature representation
• Linear Representation Hypothesis
• spurious features
• latent space

Innovation

• Linear Centroids Hypothesis
• feature interpretability
• sparse autoencoders
• vision transformers
• neural circuits