Faithfulness to Refusal: A Causal Audit of Neuron Selectors

2026-07-06

AI Summary

This work addresses the lack of direct validation regarding whether existing neuron attribution methods genuinely identify neurons causally important to model behavior. The authors propose a paired causal auditing framework based on one-shot zero-ablation interventions, integrating contrastive evaluations of harmful versus harmless behaviors with randomized controlled trials to systematically assess attribution efficacy across five large language models. Results demonstrate that the evaluated methods can effectively instill safety-aligned refusal capabilities with low benign rejection rates while preserving linguistic fluency. Notably, the sets of neurons activated by different attribution methods exhibit minimal overlap, suggesting that refusal mechanisms reside within redundant subspaces. Furthermore, high ranking stability does not guarantee strong causal validity, underscoring the necessity of explicit causal verification for attribution techniques.

Abstract

Attribution scores increasingly identify which neuron rows of a language model matter for applications such as pruning, interpretability, and editing for safety, yet whether they identify causally important rows is rarely tested directly. We address this with two paired audits built on one-shot neuron-row zeroing. We first audit selectors at the language-modeling level: attribution methods substantially outperform activation and magnitude-based baselines at identifying dispensable rows across five LLMs. We then adapt the same intervention into a behavior test by driving it with a contrastive harmful-versus-benign signal; the attributed rows are sufficient to install refusal on hate and crime while keeping benign over-refusal low and preserving language model fluency, and specific in that layer-matched random controls at the same depths fail. Highly rank-stable selectors can be among the least causally valid. Refusal moreover lives in a redundant subspace, where different attribution methods install it through largely disjoint row sets, so the recovered edit is one realization of a sufficient set rather than a unique mechanism. Together, these findings show that rank-stability proxies miss the kinds of selector failures a direct causal audit can surface.%

Problem

• faithfulness
• causal audit
• neuron selectors
• attribution scores
• language models

Innovation

• causal audit
• neuron selector
• attribution methods
• refusal behavior
• redundant subspace