🤖 AI Summary
This work addresses the challenge of systematically comparing high-dimensional, unstructured data generated by large-scale activation patching, which hinders scalable mechanistic interpretability of causal circuits in large language models. The authors propose modeling activation patching results as graph structures and introduce three graph construction methods based on causal mediation, partial correlation, and co-influence. To distinguish task-specific signals from general causal evidence, they incorporate rigorous baseline controls. Evaluating on the Indirect Object Identification (IOI) task in GPT-2 Small using graph kernels and feature extraction, they find that local edge-slot features outperform global graph descriptors in classification performance. Moreover, edges selected via causal inference (CI) and partial correlation (PC) criteria significantly surpass random or low-rank baselines, demonstrating that graph representations effectively compress and reveal structured causal signals.
📝 Abstract
Mechanistic interpretability aims to reverse-engineer transformer computations by identifying causal circuits through activation patching. However, scaling these interventions across diverse prompts and task families produces high-dimensional, unstructured datasets that are difficult to compare systematically. We propose a framework that reframes mechanistic analysis as a graph machine-learning problem by representing activation-patching profiles as patch-effect graphs over model components. We introduce three graph-construction methods: direct-influence via causal mediation, partial-correlation, and co-influence and apply graph kernels to analyze the resulting structures. Evaluating this approach on GPT-2 Small using Indirect Object Identification (IOI) and related tasks, we find that patch-effect graphs preserve discriminative structural signals. Specifically, localized edge-slot features provide higher classification accuracy than global graph-shape descriptors. A screened paired-patching validation suggests that CI and PC selected candidate edges correspond to stronger activation-influence effects than random or low-rank candidates. Crucially, by evaluating these representations against rigorous prompt-only and raw patch-effect controls, we make the evidential scope of the benchmark explicit: graph features compress structured patching signal, while raw tensors and surface cues define strong baselines that any circuit-level claim should address. Ultimately, our framework provides a compression and evaluation pipeline for comparing patching-derived structures under controlled baselines, separating robust slice-discriminative evidence from stronger task-general causal-circuit claims.