Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models

2026-08-09

AI Summary

This work addresses the limitation of existing activation steering methods, which rely on fixed global injection layers and struggle to adapt to the optimal intervention location for diverse inputs. The authors propose a deployable, instance-wise dynamic multi-layer steering approach that predicts, based on prompt embeddings, the optimal steering layer and direction for each input and modulates steering depth via an adaptive gating mechanism. This method is the first to enable unsupervised, label-free, instance-specific multi-layer steering and introduces a “direction-over-magnitude” principle to explain and mitigate behavioral flipping and fluency collapse. Experiments on two 8B open-source models across six personality traits demonstrate that the approach closely approximates oracle performance, consistently matches or exceeds the non-steered baseline on average, and effectively avoids output degradation commonly induced by high-layer steering.

Abstract

Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task. We argue that the best layers are an instance-level decision, and we make per-instance, multi-layer selection both well understood and deployable. On two open-weight 8B models and six binary persona traits, a per-instance oracle over layer subsets shows that the best layers vary from one input to the next: on most trait-model pairs, no fixed global layer set recovers the per-instance benefit. A greedy rule that ranks layers by single-layer marginal effect recovers nearly all of the oracle's benefit, but both must score candidate layers against the gold answer, so neither can run at deployment; the rule instead becomes the target a prompt-only predictor is trained to reproduce. Our deployable recipe needs no label at inference: a per-instance layer ranker read off the prompt embedding, a classifier that infers the steering direction, and an adaptive gate that scores short steered passes against that inferred direction and steers no more layers than necessary. The recipe recovers most of the oracle's lift (the bulk on the stronger model, a clear majority on the harder one), never drives any trait-model pair below its unsteered alignment baseline on average, and largely avoids the fluency collapse that strong global selection incurs at higher layer counts. A mechanistic account, "direction over magnitude", explains the behavioural flip under a mis-directed global set, the output collapse from steering too many layers, and the ceiling of unsteerable inputs.

Problem

• activation steering
• layer selection
• instance-level adaptation
• language model behavior
• residual stream

Innovation

• activation steering
• per-instance layer selection
• deployable intervention
• direction over magnitude
• residual stream editing