🤖 AI Summary
Existing methods struggle to predict where semantic vector interventions will be effective across concepts and model layers. This work proposes the Linear Accessibility Profile (LAP), a training-free layer diagnostic metric, \(A_{\text{lin}}\), that integrates the logit lens, unembedding matrices, and mean-difference analysis to accurately forecast both intervention efficacy and the optimal layer for intervention—achieving this capability for the first time. Grounded in a three-mechanism framework, LAP delineates the conditions under which linear semantic vectors succeed, require nonlinear alternatives, or fail entirely. Experiments across five models and 24 binary concept pairs show that \(A_{\text{lin}}\) correlates strongly (0.86–0.91) with actual intervention outcomes, and that layers selected by LAP significantly outperform default middle layers, yielding end-to-end improvements validated on Gemma-2-2B and OLMo-2-1B-Instruct.
📝 Abstract
Steering vectors work for some concepts and layers but fail for others, and practitioners have no way to predict which setting applies before running an intervention. We introduce the Linear Accessibility Profile (LAP), a per-layer diagnostic that repurposes the logit lens as a predictor of steering vector effectiveness. The key measure, $A_{\mathrm{lin}}$, applies the model's unembedding matrix to intermediate hidden states, requiring no training. Across 24 controlled binary concept families on five models (Pythia-2.8B to Llama-8B), peak $A_{\mathrm{lin}}$ predicts steering effectiveness at $ρ= +0.86$ to $+0.91$ and layer selection at $ρ= +0.63$ to $+0.92$. A three-regime framework explains when difference-of-means steering works, when nonlinear methods are needed, and when no method can work. An entity-steering demo confirms the prediction end-to-end: steering at the LAP-recommended layer redirects completions on Gemma-2-2B and OLMo-2-1B-Instruct, while the middle layer (the standard heuristic) has no effect on either model.