Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol

2026-06-30

AI Summary

Current approaches to mechanistic interpretability lack a unified, structured representation, hindering reusability and queryability. This work proposes the Manifestation Units protocol, which standardizes interpretability findings of neural network components into evaluable representational bottlenecks through typed tuples (E, S, R, D, G) and attention-head primitives (T). The protocol enables cross-architecture generality, hybrid natural language retrieval, and causal validation. Empirical evaluations across multiple models demonstrate significant advantages over unstructured baselines: it verifies that CNN filters satisfy causal necessity and sufficiency, successfully reproduces IOI circuit membership, and identifies the core irreducible subspace S+R.

Abstract

Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact. Their outputs, however, are not easily reusable: selectivity tables, circuit diagrams, and feature lists remain locked in per-study notebooks - non-composable, not queryable in natural language, and not directly actionable for downstream audit or intervention. We study the representation layer that sits between these analyses and downstream use as a bottleneck that can be evaluated independently, and introduce Manifestation Units, a typed tuple protocol (E, S, R, D, G) extended with attention-head primitives (T) for transformer architectures, organising per-component statistics into structured fields populated automatically and queried through hybrid retrieval. Instantiated across generative vision (beta-VAE), discriminative vision (CNN), and language (GPT-2), the protocol supports two findings: typed structure substantially outperforms unstructured baselines on retrieval, and CNN filters retrieved by the schema satisfy causal sufficiency and necessity criteria under matched-budget controls. The schema absorbs attention-head primitives without modification, set-recovers known IOI circuit members under retrieval-budget-matched controls, and reveals an irreducible two-field core (S+R) with remaining fields either redundant or actively interfering. We present this as schema infrastructure for mechanistic interpretability rather than frontier-scale validation.

Problem

• mechanistic interpretability
• representation bottleneck
• reusability
• structured representation
• downstream audit

Innovation

• Manifestation Units
• mechanistic interpretability
• structured representation
• retrieval schema
• causal sufficiency