🤖 AI Summary
Current language models lack transparency in their mechanisms for rejecting harmful requests, and prevailing intervention techniques often degrade output coherence under strong intervention intensities. This work proposes Contrastive Neuron Attribution (CNA), a method that precisely identifies, during the forward pass, the key MLP neurons responsible for distinguishing harmful from benign inputs and applies targeted modulation to steer model behavior. The study reveals, for the first time, that alignment fine-tuning transforms the inherent discriminative structures of base models into sparse, localized rejection gating mechanisms. Requiring neither gradients nor additional training, CNA enables cross-architecture analysis (e.g., Llama/Qwen) and reduces refusal rates by over 50% on standard jailbreaking benchmarks while preserving output fluency and non-degeneracy across all intervention strengths.
📝 Abstract
Language models are instruction-tuned to refuse harmful requests, but the mechanisms underlying this behavior remain poorly understood. Popular steering methods operate on the residual stream and degrade output coherence at high intervention strengths, limiting their practical use. We introduce contrastive neuron attribution (CNA), which identifies the 0.1% of MLP neurons whose activations most distinguish harmful from benign prompts, requiring only forward passes with no gradients or auxiliary training. In instruct models, ablating the discovered circuit reduces refusal rates by over 50% on a standard jailbreak benchmark while preserving fluency and non-degeneracy across all steering strengths. Applying CNA to matched base and instruct models across Llama and Qwen architectures (from 1B to 72B parameters), we find that base models contain similar late-layer discrimination structures but steering these neurons produces only content shifts, not behavioral change. These results demonstrate that neuron-level intervention enables reliable behavioral steering without the quality tradeoffs of residual-stream methods. More broadly, our findings suggest that alignment fine-tuning transforms pre-existing discrimination structure into a sparse, targetable refusal gate.